[GitHub] [airflow] Lee-W commented on pull request #34108: Refactor: Consolidate import and usage of random

2023-09-07 Thread via GitHub
Lee-W commented on PR #34108: URL: https://github.com/apache/airflow/pull/34108#issuecomment-1709599113 > > Same as #34113. I'm wondering why we want to consolidate imports this way. Does that mean we should avoid `from ... import ...` pattern. > > I'm suggesting having all imported m

[GitHub] [airflow] Lee-W commented on pull request #34113: Refactor: Consolidate 'import textwrap'

2023-09-07 Thread via GitHub
Lee-W commented on PR #34113: URL: https://github.com/apache/airflow/pull/34113#issuecomment-1709599439 > > I'm a bit curious about why we want to do it this way. Also, would it make sense if we document these changes somewhere in the contribution document so that newcomers know what to fol

[GitHub] [airflow] A-Costa commented on pull request #34137: Add use_glob to GCSObjectExistenceSensor

2023-09-07 Thread via GitHub
A-Costa commented on PR #34137: URL: https://github.com/apache/airflow/pull/34137#issuecomment-1709608581 Sure, that was just an initial draft. I will try to push a complete version of this PR by the end of the day -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [airflow] uranusjr commented on a diff in pull request #32646: Add task context logging feature to allow forwarding messages to task logs

2023-09-07 Thread via GitHub
uranusjr commented on code in PR #32646: URL: https://github.com/apache/airflow/pull/32646#discussion_r1318194790 ## airflow/models/taskinstance.py: ## @@ -895,6 +895,19 @@ def refresh_from_db(self, session: Session = NEW_SESSION, lock_for_update: bool else:

[GitHub] [airflow] uranusjr commented on a diff in pull request #32646: Add task context logging feature to allow forwarding messages to task logs

2023-09-07 Thread via GitHub
uranusjr commented on code in PR #32646: URL: https://github.com/apache/airflow/pull/32646#discussion_r1318194790 ## airflow/models/taskinstance.py: ## @@ -895,6 +895,19 @@ def refresh_from_db(self, session: Session = NEW_SESSION, lock_for_update: bool else:

[GitHub] [airflow] uranusjr commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-09-07 Thread via GitHub
uranusjr commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1318200546 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -87,35 +131,231 @@ def is_logged_in(self) -> bool: """Return whether the user is logged in."""

[GitHub] [airflow] uranusjr commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-09-07 Thread via GitHub
uranusjr commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1318202392 ## airflow/auth/managers/models/resource_method.py: ## @@ -0,0 +1,38 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licen

[GitHub] [airflow] uranusjr commented on pull request #34138: Fix mapped task not waiting for upstream mapped to complete

2023-09-07 Thread via GitHub
uranusjr commented on PR #34138: URL: https://github.com/apache/airflow/pull/34138#issuecomment-1709634038 I think waiting for all upstreams to finish first is the correct behaviour. The documentation also hinted on this. [In the first example](https://airflow.apache.org/docs/apache-airflow

[GitHub] [airflow] uranusjr commented on a diff in pull request #34118: Refactor: Simplify string generation

2023-09-07 Thread via GitHub
uranusjr commented on code in PR #34118: URL: https://github.com/apache/airflow/pull/34118#discussion_r1318222825 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -166,9 +167,8 @@ def hook(self): def path_to_s3_dataset(path) -> Dataset: from openlineag

[GitHub] [airflow] Bowrna closed pull request #31996: heartbeat logmessage fix

2023-09-07 Thread via GitHub
Bowrna closed pull request #31996: heartbeat logmessage fix URL: https://github.com/apache/airflow/pull/31996 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[GitHub] [airflow] uranusjr commented on a diff in pull request #34118: Refactor: Simplify string generation

2023-09-07 Thread via GitHub
uranusjr commented on code in PR #34118: URL: https://github.com/apache/airflow/pull/34118#discussion_r1318228119 ## scripts/in_container/run_migration_reference.py: ## @@ -135,22 +135,18 @@ def update_docs(revisions: Iterable[Script]): ) -def num_to_prefix(idx: int) ->

[GitHub] [airflow] ephraimbuddy commented on pull request #34138: Fix mapped task not waiting for upstream mapped to complete

2023-09-07 Thread via GitHub
ephraimbuddy commented on PR #34138: URL: https://github.com/apache/airflow/pull/34138#issuecomment-1709660867 > I think waiting for all upstreams to finish first is the correct behaviour. The documentation also hinted on this. [In the first example](https://airflow.apache.org/docs/apache-a

[GitHub] [airflow] Lee-W opened a new pull request, #34156: respect soft_fail argument when exception is raised for airbyte sensors

2023-09-07 Thread via GitHub
Lee-W opened a new pull request, #34156: URL: https://github.com/apache/airflow/pull/34156 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideline

[GitHub] [airflow] Bowrna commented on pull request #31996: heartbeat logmessage fix

2023-09-07 Thread via GitHub
Bowrna commented on PR #31996: URL: https://github.com/apache/airflow/pull/31996#issuecomment-1709661764 I wrongly did a git force push with main branch changes. Please excuse. I will clean up and commit the changes -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] Lee-W opened a new pull request, #34157: respect soft_fail argument when exception is raised for alibaba sensors

2023-09-07 Thread via GitHub
Lee-W opened a new pull request, #34157: URL: https://github.com/apache/airflow/pull/34157 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**

[GitHub] [airflow] uranusjr commented on pull request #34138: Fix mapped task not waiting for upstream mapped to complete

2023-09-07 Thread via GitHub
uranusjr commented on PR #34138: URL: https://github.com/apache/airflow/pull/34138#issuecomment-1709683277 It makes sense for `task3` to skip `task4` in this case, but the impression I got from your description above is if `task[1]` finishes first (with success), the downstream task would r

[GitHub] [airflow] dstandish commented on pull request #34138: Fix mapped task not waiting for upstream mapped to complete

2023-09-07 Thread via GitHub
dstandish commented on PR #34138: URL: https://github.com/apache/airflow/pull/34138#issuecomment-1709696696 > That’s the part that I think is wrong, becauase for normal tasks, a SUCCESS `task1` would not trigger `task4` immediately. Failure / skip is different because "fast fail" or "

[GitHub] [airflow] uranusjr commented on pull request #34138: Fix mapped task not waiting for upstream mapped to complete

2023-09-07 Thread via GitHub
uranusjr commented on PR #34138: URL: https://github.com/apache/airflow/pull/34138#issuecomment-1709721775 Yeah and from the initial description that is a thing for mapped tasks right now? -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [airflow] mziwisky opened a new issue, #34158: cannot use `get_current_context()` in a `@task.virtualenv` task

2023-09-07 Thread via GitHub
mziwisky opened a new issue, #34158: URL: https://github.com/apache/airflow/issues/34158 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened On Airflow 2.5.1 (on AWS MWAA), I ran this DAG: ```python from datetime import da

[GitHub] [airflow] Bowrna opened a new pull request, #34159: committing old changes

2023-09-07 Thread via GitHub
Bowrna opened a new pull request, #34159: URL: https://github.com/apache/airflow/pull/34159 closes: https://github.com/apache/airflow/issues/31810 relates: https://github.com/apache/airflow/issues/31810 This is the work done in the PR https://github.com/apache/airflow/pull/31996

[GitHub] [airflow] Abhishek-kumar-samsung commented on issue #34058: UI Grid error when DAG has been removed

2023-09-07 Thread via GitHub
Abhishek-kumar-samsung commented on issue #34058: URL: https://github.com/apache/airflow/issues/34058#issuecomment-1709782180 > > I ran using breeze start-airflow, it won't occur there? > > You can add `--dev-mode` flag when you run `start-airflow` - then instead of the "ooops" screen

[GitHub] [airflow] A-Costa commented on a diff in pull request #34137: Add use_glob to GCSObjectExistenceSensor

2023-09-07 Thread via GitHub
A-Costa commented on code in PR #34137: URL: https://github.com/apache/airflow/pull/34137#discussion_r1318321804 ## airflow/providers/google/cloud/triggers/gcs.py: ## @@ -98,9 +102,14 @@ async def _object_exists(self, hook: GCSAsyncHook, bucket_name: str, object_name a

[GitHub] [airflow] mobuchowski merged pull request #33959: openlineage, dbt: add OpenLineage support for DBT Cloud

2023-09-07 Thread via GitHub
mobuchowski merged PR #33959: URL: https://github.com/apache/airflow/pull/33959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

[airflow] branch main updated: Add OpenLineage support for DBT Cloud. (#33959)

2023-09-07 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5eea4e632c Add OpenLineage support for DBT Clo

[GitHub] [airflow] Bowrna closed pull request #34159: committing old changes

2023-09-07 Thread via GitHub
Bowrna closed pull request #34159: committing old changes URL: https://github.com/apache/airflow/pull/34159 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

[GitHub] [airflow] Bowrna opened a new pull request, #34160: heartbeat failure log message fix

2023-09-07 Thread via GitHub
Bowrna opened a new pull request, #34160: URL: https://github.com/apache/airflow/pull/34160 closes: https://github.com/apache/airflow/issues/31810 relates: https://github.com/apache/airflow/issues/31810 This is the work done in the PR https://github.com/apache/airflow/pull/31996

[GitHub] [airflow] Bowrna commented on pull request #34160: heartbeat failure log message fix

2023-09-07 Thread via GitHub
Bowrna commented on PR #34160: URL: https://github.com/apache/airflow/pull/34160#issuecomment-1709804600 @potiuk this is the old PR. created a new PR with changes as the old one is closed by mistake -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [airflow] potiuk commented on pull request #34108: Refactor: Consolidate import and usage of random

2023-09-07 Thread via GitHub
potiuk commented on PR #34108: URL: https://github.com/apache/airflow/pull/34108#issuecomment-1709806138 I think those kind of rules are rarely read and we have two approaches there that 'work' a) hope that people will follow what is in the code (not perfect but mostly works) - this

[GitHub] [airflow] Lee-W opened a new pull request, #34161: respect soft_fail argument when exception is raised for ftp sensors

2023-09-07 Thread via GitHub
Lee-W opened a new pull request, #34161: URL: https://github.com/apache/airflow/pull/34161 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**

[GitHub] [airflow] potiuk commented on issue #34158: cannot use `get_current_context()` in a `@task.virtualenv` task

2023-09-07 Thread via GitHub
potiuk commented on issue #34158: URL: https://github.com/apache/airflow/issues/34158#issuecomment-1709848083 I think it would be possible, it's just a matter of serializing the context and adding code in https://github.com/apache/airflow/blob/main/airflow/utils/python_virtualenv_script.jin

[GitHub] [airflow] nathadfield opened a new issue, #34162: `BigQueryInsertJobOperator` doesn't handle job errors caused by running tasks in deferrable mode

2023-09-07 Thread via GitHub
nathadfield opened a new issue, #34162: URL: https://github.com/apache/airflow/issues/34162 ### Apache Airflow version 2.7.0 ### What happened When a BigQuery job running in deferrable ends before completion - i.e. cancelled via the UI or terminated due a lack of resourc

[GitHub] [airflow] bbovenzi merged pull request #34136: Ensure details panel is shown when any tab is selected

2023-09-07 Thread via GitHub
bbovenzi merged PR #34136: URL: https://github.com/apache/airflow/pull/34136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[airflow] branch main updated: Ensure details panel is shown when any tab is selected (#34136)

2023-09-07 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0b319e79ec Ensure details panel is shown when any

[GitHub] [airflow] Lee-W opened a new pull request, #34163: respect soft_fail argument when exception is raised for tableau sensors

2023-09-07 Thread via GitHub
Lee-W opened a new pull request, #34163: URL: https://github.com/apache/airflow/pull/34163 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guide

[GitHub] [airflow] Seokyun-Ha commented on pull request #34071: Databricks ClusteState & Clsuter GET API

2023-09-07 Thread via GitHub
Seokyun-Ha commented on PR #34071: URL: https://github.com/apache/airflow/pull/34071#issuecomment-1709881948 Hello, @Lee-W @hussein-awala, We've applied all your feedbacks! 😄 Also, we implemented mock testing code as well. Please, take a look 🙏 Thanks! -- This is an automated message fro

[GitHub] [airflow] Lee-W opened a new pull request, #34164: respect soft_fail argument when exception is raised for dbt sensors

2023-09-07 Thread via GitHub
Lee-W opened a new pull request, #34164: URL: https://github.com/apache/airflow/pull/34164 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guide

[GitHub] [airflow] Lee-W opened a new pull request, #34165: respect soft_fail argument when exception is raised for google-marketing-platform sensors

2023-09-07 Thread via GitHub
Lee-W opened a new pull request, #34165: URL: https://github.com/apache/airflow/pull/34165 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guide

[GitHub] [airflow] ephraimbuddy commented on pull request #34138: Fix mapped task not waiting for upstream mapped to complete

2023-09-07 Thread via GitHub
ephraimbuddy commented on PR #34138: URL: https://github.com/apache/airflow/pull/34138#issuecomment-1709915497 > Yeah and from the initial description that is a thing for mapped tasks right now? The initial description is wrong for using `success` state. It happens in the case of fai

[GitHub] [airflow] potiuk commented on issue #33869: airflow tasks test triggers a dag run with a run_id like: __airflow_temporary_run_2023-08-29T07:39:51.001501+00:00__ but doesn't delete it when don

2023-09-07 Thread via GitHub
potiuk commented on issue #33869: URL: https://github.com/apache/airflow/issues/33869#issuecomment-1709917907 CAnnot reproduce. Generally the temporary task task run should be removed when the `test` command completes. I think you must be doing or expecting something different. And

[GitHub] [airflow] potiuk closed issue #33869: airflow tasks test triggers a dag run with a run_id like: __airflow_temporary_run_2023-08-29T07:39:51.001501+00:00__ but doesn't delete it when done

2023-09-07 Thread via GitHub
potiuk closed issue #33869: airflow tasks test triggers a dag run with a run_id like: __airflow_temporary_run_2023-08-29T07:39:51.001501+00:00__ but doesn't delete it when done URL: https://github.com/apache/airflow/issues/33869 -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] potiuk merged pull request #34161: respect soft_fail argument when exception is raised for ftp sensors

2023-09-07 Thread via GitHub
potiuk merged PR #34161: URL: https://github.com/apache/airflow/pull/34161 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: fix(providers/ftp): respect soft_fail argument when exception is raised (#34161)

2023-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4f0e4254e4 fix(providers/ftp): respect soft_fail ar

[GitHub] [airflow] Lee-W opened a new pull request, #34167: respect soft_fail argument when exception is raised for spark-kubernetes sensors

2023-09-07 Thread via GitHub
Lee-W opened a new pull request, #34167: URL: https://github.com/apache/airflow/pull/34167 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideline

[GitHub] [airflow] DjVinnii commented on issue #34147: Update docs to mark Pluggy interface as stable rather than experimental

2023-09-07 Thread via GitHub
DjVinnii commented on issue #34147: URL: https://github.com/apache/airflow/issues/34147#issuecomment-1709930431 Is this also relevant for the [Listeners](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/listeners.html) or is that a separate topic? As these

[GitHub] [airflow] tiranux opened a new pull request, #34168: Validate DAG task_instance.priority_weight column overflow value during import

2023-09-07 Thread via GitHub
tiranux opened a new pull request, #34168: URL: https://github.com/apache/airflow/pull/34168 Closes: [#22781](https://github.com/apache/airflow/issues/22781) This is a follow up implementation of fix during DAG import regarding https://github.com/apache/airflow/pull/22

[GitHub] [airflow] Lee-W opened a new pull request, #34169: respect soft_fail argument when exception is raised for sftp sensors

2023-09-07 Thread via GitHub
Lee-W opened a new pull request, #34169: URL: https://github.com/apache/airflow/pull/34169 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**

[GitHub] [airflow] tiranux commented on pull request #34168: Validate DAG task_instance.priority_weight column overflow value during import

2023-09-07 Thread via GitHub
tiranux commented on PR #34168: URL: https://github.com/apache/airflow/pull/34168#issuecomment-1709956072 Adding @uranusjr to conversation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [airflow] aryehgigi opened a new issue, #34170: ComputeEngineInsertInstanceOperator doesn't respect jinja-templated instance name when given in body argument

2023-09-07 Thread via GitHub
aryehgigi opened a new issue, #34170: URL: https://github.com/apache/airflow/issues/34170 ### Apache Airflow version 2.7.0 ### What happened when initializing the operator with a `body` that has a jinja-templated `name` field and without a `resource_id`, the creation fai

[GitHub] [airflow] boring-cyborg[bot] commented on issue #34170: ComputeEngineInsertInstanceOperator doesn't respect jinja-templated instance name when given in body argument

2023-09-07 Thread via GitHub
boring-cyborg[bot] commented on issue #34170: URL: https://github.com/apache/airflow/issues/34170#issuecomment-1709970170 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[airflow] branch constraints-main updated: Updating constraints. Github run id:6108659616

2023-09-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 92190a4d11 Updating con

[GitHub] [airflow] ephraimbuddy commented on issue #34023: Trigger Rule ONE_FAILED does not work in task group with mapped tasks

2023-09-07 Thread via GitHub
ephraimbuddy commented on issue #34023: URL: https://github.com/apache/airflow/issues/34023#issuecomment-1710005257 > Ok. Found a fix for this but still need to reason more about it: > > ```diff > diff --git a/airflow/models/taskinstance.py b/airflow/models/taskinstance.py > ind

[GitHub] [airflow] Taragolis commented on a diff in pull request #34168: Validate DAG task_instance.priority_weight column overflow value during import

2023-09-07 Thread via GitHub
Taragolis commented on code in PR #34168: URL: https://github.com/apache/airflow/pull/34168#discussion_r1318498531 ## airflow/utils/dag_parameters_overflow.py: ## @@ -0,0 +1,56 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreem

[GitHub] [airflow] Taragolis commented on a diff in pull request #34168: Validate DAG task_instance.priority_weight column overflow value during import

2023-09-07 Thread via GitHub
Taragolis commented on code in PR #34168: URL: https://github.com/apache/airflow/pull/34168#discussion_r1318498531 ## airflow/utils/dag_parameters_overflow.py: ## @@ -0,0 +1,56 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreem

[GitHub] [airflow] aryehgigi opened a new pull request, #34171: fix

2023-09-07 Thread via GitHub
aryehgigi opened a new pull request, #34171: URL: https://github.com/apache/airflow/pull/34171 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideline

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #34171: fix

2023-09-07 Thread via GitHub
boring-cyborg[bot] commented on PR #34171: URL: https://github.com/apache/airflow/pull/34171#issuecomment-1710025081 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] Taragolis commented on a diff in pull request #34168: Validate DAG task_instance.priority_weight column overflow value during import

2023-09-07 Thread via GitHub
Taragolis commented on code in PR #34168: URL: https://github.com/apache/airflow/pull/34168#discussion_r1318498531 ## airflow/utils/dag_parameters_overflow.py: ## @@ -0,0 +1,56 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreem

[GitHub] [airflow] potiuk merged pull request #34167: respect soft_fail argument when exception is raised for spark-kubernetes sensors

2023-09-07 Thread via GitHub
potiuk merged PR #34167: URL: https://github.com/apache/airflow/pull/34167 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: fix(providers/spark-kubernetes): respect soft_fail argument when exception is raised (#34167)

2023-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6e1de4f9b9 fix(providers/spark-kubernetes): respect

[GitHub] [airflow] potiuk merged pull request #34164: respect soft_fail argument when exception is raised for dbt sensors

2023-09-07 Thread via GitHub
potiuk merged PR #34164: URL: https://github.com/apache/airflow/pull/34164 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated (6e1de4f9b9 -> 0ecbbacd21)

2023-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 6e1de4f9b9 fix(providers/spark-kubernetes): respect soft_fail argument when exception is raised (#34167) add 0ecb

[GitHub] [airflow] potiuk merged pull request #34169: respect soft_fail argument when exception is raised for sftp sensors

2023-09-07 Thread via GitHub
potiuk merged PR #34169: URL: https://github.com/apache/airflow/pull/34169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: fix(providers/sftp): respect soft_fail argument when exception is raised (#34169)

2023-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f5857a9b8f fix(providers/sftp): respect soft_fail a

[GitHub] [airflow] potiuk merged pull request #34165: respect soft_fail argument when exception is raised for google-marketing-platform sensors

2023-09-07 Thread via GitHub
potiuk merged PR #34165: URL: https://github.com/apache/airflow/pull/34165 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: fix(providers/google-marketing-platform): respect soft_fail argument when exception is raised (#34165)

2023-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ff23a30e5b fix(providers/google-marketing-platform)

[GitHub] [airflow] ahzaz opened a new pull request, #34172: Add support for driver pool, instance flexibility policy, and min_num_instances for Dataproc

2023-09-07 Thread via GitHub
ahzaz opened a new pull request, #34172: URL: https://github.com/apache/airflow/pull/34172 --- Add following to ClusterGenerator for Dataproc - min_num_workers: The minimum number of primary worker instances to create - driver_pool_size: The size of the cluster dr

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #34172: Add support for driver pool, instance flexibility policy, and min_num_instances for Dataproc

2023-09-07 Thread via GitHub
boring-cyborg[bot] commented on PR #34172: URL: https://github.com/apache/airflow/pull/34172#issuecomment-1710070518 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] boring-cyborg[bot] commented on issue #34173: FacebookAdsReportToGcsOperator method _flush_rows() infers field names from first data point instead of declared fields

2023-09-07 Thread via GitHub
boring-cyborg[bot] commented on issue #34173: URL: https://github.com/apache/airflow/issues/34173#issuecomment-1710092248 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[GitHub] [airflow] Taishan314 opened a new issue, #34173: FacebookAdsReportToGcsOperator method _flush_rows() infers field names from first data point instead of declared fields

2023-09-07 Thread via GitHub
Taishan314 opened a new issue, #34173: URL: https://github.com/apache/airflow/issues/34173 ### Apache Airflow version 2.7.0 ### What happened I created a task to retrieve insight level ad data using the FacebookAdsReportToGcsOperator. Whilst running the pipeline, the dag

[GitHub] [airflow] DFINITYManu commented on issue #28647: Intermitent log on deferrable operator

2023-09-07 Thread via GitHub
DFINITYManu commented on issue #28647: URL: https://github.com/apache/airflow/issues/28647#issuecomment-1710106822 I just upgraded to 2.7.0 to see if the issue was addressed, and I'm still experiencing the same issue of not being able to fetch logs from deferred tasks: ![image](https

[GitHub] [airflow] syun64 commented on pull request #34126: New 'cleared' boolean attribute for dag_run

2023-09-07 Thread via GitHub
syun64 commented on PR #34126: URL: https://github.com/apache/airflow/pull/34126#issuecomment-1710114384 > We already have `try_number` on TaskInstance for a similar purpose (you can clear and rerun one single task), I wonder if there’s some unifying can be done here. Yeah, that was

[GitHub] [airflow] Taragolis commented on a diff in pull request #34168: Validate DAG task_instance.priority_weight column overflow value during import

2023-09-07 Thread via GitHub
Taragolis commented on code in PR #34168: URL: https://github.com/apache/airflow/pull/34168#discussion_r1318575928 ## airflow/utils/dag_parameters_overflow.py: ## @@ -0,0 +1,56 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreem

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #34174: Update cluster-policies.rst

2023-09-07 Thread via GitHub
boring-cyborg[bot] commented on PR #34174: URL: https://github.com/apache/airflow/pull/34174#issuecomment-1710133299 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] TheCodyRich opened a new pull request, #34174: Update cluster-policies.rst

2023-09-07 Thread via GitHub
TheCodyRich opened a new pull request, #34174: URL: https://github.com/apache/airflow/pull/34174 Removed experimental tag from Pluggy interface. Resolves #34147 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://gi

[GitHub] [airflow] TheCodyRich commented on issue #34147: Update docs to mark Pluggy interface as stable rather than experimental

2023-09-07 Thread via GitHub
TheCodyRich commented on issue #34147: URL: https://github.com/apache/airflow/issues/34147#issuecomment-1710134761 I would assume they would be the same, but for now I've opened a PR just for the Cluster Policies -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [airflow] potiuk commented on issue #28647: Intermitent log on deferrable operator

2023-09-07 Thread via GitHub
potiuk commented on issue #28647: URL: https://github.com/apache/airflow/issues/28647#issuecomment-1710136881 The problem is ("Name or service not known") is that your worker's address uses the local IP address that is not reachable from the webserver. The address it tries to connect

[GitHub] [airflow] rahultilokanizs opened a new issue, #34175: Autoscaling w.r.t Resource utilization with celery executor

2023-09-07 Thread via GitHub
rahultilokanizs opened a new issue, #34175: URL: https://github.com/apache/airflow/issues/34175 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Hello Team, We are using Airflow with Celery executor in which we are facing

[GitHub] [airflow] boring-cyborg[bot] commented on issue #34175: Autoscaling w.r.t Resource utilization with celery executor

2023-09-07 Thread via GitHub
boring-cyborg[bot] commented on issue #34175: URL: https://github.com/apache/airflow/issues/34175#issuecomment-1710149568 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[GitHub] [airflow] vincbeck commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-09-07 Thread via GitHub
vincbeck commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1318619327 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -87,35 +131,231 @@ def is_logged_in(self) -> bool: """Return whether the user is logged in."""

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #32646: Add task context logging feature to allow forwarding messages to task logs

2023-09-07 Thread via GitHub
ephraimbuddy commented on code in PR #32646: URL: https://github.com/apache/airflow/pull/32646#discussion_r1318621569 ## airflow/utils/log/task_context_logger.py: ## @@ -0,0 +1,120 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

[GitHub] [airflow] Taragolis commented on pull request #34174: Update cluster-policies.rst

2023-09-07 Thread via GitHub
Taragolis commented on PR #34174: URL: https://github.com/apache/airflow/pull/34174#issuecomment-1710181098 I've just wondering should Vote or at least [Lazy Consensus](https://community.apache.org/committers/lazyConsensus.html#stating-lazy-consensus) created in [dev-list](https://airflow.a

[GitHub] [airflow] vincbeck commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-09-07 Thread via GitHub
vincbeck commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1318645914 ## airflow/auth/managers/models/resource_method.py: ## @@ -0,0 +1,38 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licen

[GitHub] [airflow] vincbeck closed pull request #33213: Implement `is_authorized()` in auth manager

2023-09-07 Thread via GitHub
vincbeck closed pull request #33213: Implement `is_authorized()` in auth manager URL: https://github.com/apache/airflow/pull/33213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [airflow] vincbeck opened a new pull request, #33213: Implement `is_authorized()` in auth manager

2023-09-07 Thread via GitHub
vincbeck opened a new pull request, #33213: URL: https://github.com/apache/airflow/pull/33213 Resolves #32204. Implement the method `is_authorized()` in auth manager. The purpose of this method is to be the user authorization decision maker. The purpose of this method is to check whe

[GitHub] [airflow] nathadfield commented on issue #34162: `BigQueryInsertJobOperator` doesn't handle job errors caused by running tasks in deferrable mode

2023-09-07 Thread via GitHub
nathadfield commented on issue #34162: URL: https://github.com/apache/airflow/issues/34162#issuecomment-1710391895 The issue is that the job trigger only returns `TriggerEvent({"status": "error"})` and `execute_complete` https://github.com/apache/airflow/blob/main/airflow/providers/googl

[GitHub] [airflow] potiuk commented on pull request #34174: Update cluster-policies.rst

2023-09-07 Thread via GitHub
potiuk commented on PR #34174: URL: https://github.com/apache/airflow/pull/34174#issuecomment-1710408111 > I've just wondering should Vote or at least [Lazy Consensus](https://community.apache.org/committers/lazyConsensus.html#stating-lazy-consensus) created in [dev-list](https://airflow.ap

[GitHub] [airflow] cliebBS opened a new issue, #34177: EmrAddStepsOperator does not work with `deferrable=True`

2023-09-07 Thread via GitHub
cliebBS opened a new issue, #34177: URL: https://github.com/apache/airflow/issues/34177 ### Apache Airflow version 2.7.0 ### What happened When using the `EmrAddStepsOperator`, trying to use `deferrable=True` causes an exception from the Amazon provider: ``` [2

[GitHub] [airflow] boring-cyborg[bot] commented on issue #34177: EmrAddStepsOperator does not work with `deferrable=True`

2023-09-07 Thread via GitHub
boring-cyborg[bot] commented on issue #34177: URL: https://github.com/apache/airflow/issues/34177#issuecomment-1710438102 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[GitHub] [airflow] kimminw00 opened a new pull request, #34178: Add airflow db migrate command to migrateDatabaseJob

2023-09-07 Thread via GitHub
kimminw00 opened a new pull request, #34178: URL: https://github.com/apache/airflow/pull/34178 Add airflow db migrate command to migrateDatabaseJob Why is it needed? Newer Airflow(>= 2.7.0) versions can contain database migrations so we must run airflow db migrate to migrate our da

svn commit: r63841 - /release/airflow/2.7.1/

2023-09-07 Thread ephraimanierobi
Author: ephraimanierobi Date: Thu Sep 7 16:46:09 2023 New Revision: 63841 Log: Release Airflow 2.7.1 from 2.7.1rc2 Added: release/airflow/2.7.1/ release/airflow/2.7.1/apache-airflow-2.7.1-source.tar.gz - copied unchanged from r63840, dev/airflow/2.7.1rc2/apache-airflow-2.7.1-sourc

svn commit: r63842 - /release/airflow/2.7.0/

2023-09-07 Thread ephraimanierobi
Author: ephraimanierobi Date: Thu Sep 7 16:46:32 2023 New Revision: 63842 Log: Remove old release: 2.7.0 Removed: release/airflow/2.7.0/

[airflow] annotated tag constraints-latest updated (390d8d5253 -> 6ee9caecce)

2023-09-07 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to annotated tag constraints-latest in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag constraints-latest was modified! *** from 390d8d5253 (tag) to 6ee9caec

[airflow] annotated tag 2.7.1 updated (b8c416681c -> a8366c5828)

2023-09-07 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to annotated tag 2.7.1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag 2.7.1 was modified! *** from b8c416681c (commit) to a8366c5828 (tag) tagging b8c41

[airflow] annotated tag constraints-2.7.1 updated (8fa683885f -> 7bb0a67403)

2023-09-07 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to annotated tag constraints-2.7.1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag constraints-2.7.1 was modified! *** from 8fa683885f (commit) to 7bb0a67

[GitHub] [airflow] fgalind1 commented on pull request #31066: Support naming customization on helm chart resources

2023-09-07 Thread via GitHub
fgalind1 commented on PR #31066: URL: https://github.com/apache/airflow/pull/31066#issuecomment-1710488820 do you guys know if there are plans to release the next helm chart 1.11.0 https://github.com/apache/airflow/milestone/77? :) -- This is an automated message from the Apache Git Servi

[GitHub] [airflow] tiranux commented on a diff in pull request #34168: Validate DAG task_instance.priority_weight column overflow value during import

2023-09-07 Thread via GitHub
tiranux commented on code in PR #34168: URL: https://github.com/apache/airflow/pull/34168#discussion_r1318902232 ## airflow/utils/dag_parameters_overflow.py: ## @@ -0,0 +1,56 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreemen

[GitHub] [airflow] jedcunningham commented on pull request #31066: Support naming customization on helm chart resources

2023-09-07 Thread via GitHub
jedcunningham commented on PR #31066: URL: https://github.com/apache/airflow/pull/31066#issuecomment-1710513472 Yes, very soon now that 2.7.1 is out. I plan to start on it tomorrow or next week. -- This is an automated message from the Apache Git Service. To respond to the message, please

[airflow-site] branch 2.7.1-docs created (now 6592c3118f)

2023-09-07 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch 2.7.1-docs in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 6592c3118f Add documentation for Apache Airflow 2.7.1 This branch includes the following new commi

[GitHub] [airflow-site] ephraimbuddy opened a new pull request, #860: Add documentation for Apache Airflow 2.7.1

2023-09-07 Thread via GitHub
ephraimbuddy opened a new pull request, #860: URL: https://github.com/apache/airflow-site/pull/860 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[GitHub] [airflow] Abhishek-kumar-samsung opened a new pull request, #34179: correcting wrong time showing in grid view

2023-09-07 Thread via GitHub
Abhishek-kumar-samsung opened a new pull request, #34179: URL: https://github.com/apache/airflow/pull/34179 related: https://github.com/apache/airflow/issues/32751 corrected inconsistent times in grid view. I have kept the code of time on area above bars same as the time of tooltip,

[GitHub] [airflow] eumiro opened a new pull request, #34180: Refactor integer division

2023-09-07 Thread via GitHub
eumiro opened a new pull request, #34180: URL: https://github.com/apache/airflow/pull/34180 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

  1   2   >