[GitHub] [airflow] nitinmuteja commented on pull request #23531: make resources for KubernetesPodTemplate templated

2022-05-09 Thread GitBox
nitinmuteja commented on PR #23531: URL: https://github.com/apache/airflow/pull/23531#issuecomment-1120697440 Added test for the change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #23579: Update dags.rst

2022-05-09 Thread GitBox
boring-cyborg[bot] commented on PR #23579: URL: https://github.com/apache/airflow/pull/23579#issuecomment-1120789296 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] eladkal merged pull request #23579: Update dags.rst

2022-05-09 Thread GitBox
eladkal merged PR #23579: URL: https://github.com/apache/airflow/pull/23579 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Update dags.rst (#23579)

2022-05-09 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 827bfda59b Update dags.rst (#23579) 827bfda59b is

[airflow] branch constraints-main updated: Updating constraints. Build id:2293062944

2022-05-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new cca9fab8fa Updating

[GitHub] [airflow] mthakare-onshape opened a new pull request, #23579: Update dags.rst

2022-05-09 Thread GitBox
mthakare-onshape opened a new pull request, #23579: URL: https://github.com/apache/airflow/pull/23579 Missing bracket? --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #23579: Update dags.rst

2022-05-09 Thread GitBox
boring-cyborg[bot] commented on PR #23579: URL: https://github.com/apache/airflow/pull/23579#issuecomment-1120686386 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] potiuk commented on issue #23576: The xmltodict 0.13.0 breaks some emr tests

2022-05-09 Thread GitBox
potiuk commented on issue #23576: URL: https://github.com/apache/airflow/issues/23576#issuecomment-1120838998 This has been investigated by the Moto team and it turned out to be unexpected change in behaviour of `xmltodict`. They fixed it in the upcoming 3.1.9 and once they release we

[GitHub] [airflow] raphaelauv commented on pull request #23518: [FEATURE] google provider - split GkeStartPodOperator execute

2022-05-09 Thread GitBox
raphaelauv commented on PR #23518: URL: https://github.com/apache/airflow/pull/23518#issuecomment-1120889908 @kaxil hey since you work on astronomer-providers could you review , thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] github-actions[bot] commented on pull request #23575: Fix dag-processor fetch metabase config

2022-05-09 Thread GitBox
github-actions[bot] commented on PR #23575: URL: https://github.com/apache/airflow/pull/23575#issuecomment-1120704679 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the

[GitHub] [airflow] eduardchai opened a new issue, #23580: `task_id` with `.` e.g. `hello.world` is not rendered in grid view

2022-05-09 Thread GitBox
eduardchai opened a new issue, #23580: URL: https://github.com/apache/airflow/issues/23580 ### Apache Airflow version 2.3.0 (latest released) ### What happened `task_id` with `.` e.g. `hello.world` is not rendered in grid view. ### What you think should happen

[GitHub] [airflow] boring-cyborg[bot] commented on issue #23580: `task_id` with `.` e.g. `hello.world` is not rendered in grid view

2022-05-09 Thread GitBox
boring-cyborg[bot] commented on issue #23580: URL: https://github.com/apache/airflow/issues/23580#issuecomment-1120719027 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] tirkarthi commented on issue #23417: Move tree_data endpoint to REST API

2022-05-09 Thread GitBox
tirkarthi commented on issue #23417: URL: https://github.com/apache/airflow/issues/23417#issuecomment-1120813132 @bbovenzi I can take this. Please add a permalink to the github code that you want to migrate to rest API . I couldn't find `objects/tree_data` endpoint in current views. --

[GitHub] [airflow] potiuk closed issue #23572: cli command `dag-processor` uses `[core] sql_alchemy_conn`

2022-05-09 Thread GitBox
potiuk closed issue #23572: cli command `dag-processor` uses `[core] sql_alchemy_conn` URL: https://github.com/apache/airflow/issues/23572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[airflow] branch main updated: Fix dag-processor fetch metabase config (#23575)

2022-05-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9837e6d813 Fix dag-processor fetch metabase config

[GitHub] [airflow] potiuk merged pull request #23575: Fix dag-processor fetch metabase config

2022-05-09 Thread GitBox
potiuk merged PR #23575: URL: https://github.com/apache/airflow/pull/23575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #23575: Fix dag-processor fetch metabase config

2022-05-09 Thread GitBox
boring-cyborg[bot] commented on PR #23575: URL: https://github.com/apache/airflow/pull/23575#issuecomment-1120824192 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] ecerulm commented on pull request #23578: WIP - Enable kubernetes watch bookmarks. Fix 410 error

2022-05-09 Thread GitBox
ecerulm commented on PR #23578: URL: https://github.com/apache/airflow/pull/23578#issuecomment-1120692664 See #23521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] jakubno commented on a diff in pull request #23301: Fix: Exception when parsing log #20966

2022-05-09 Thread GitBox
jakubno commented on code in PR #23301: URL: https://github.com/apache/airflow/pull/23301#discussion_r867729738 ## tests/hooks/test_subprocess.py: ## @@ -96,3 +96,9 @@ def test_should_exec_subprocess(self, mock_popen, mock_temporary_directory): stderr=STDOUT,

[GitHub] [airflow] github-actions[bot] commented on pull request #23570: The output of commands of Breeze are only generated when they change

2022-05-09 Thread GitBox
github-actions[bot] commented on PR #23570: URL: https://github.com/apache/airflow/pull/23570#issuecomment-1120788500 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the

[GitHub] [airflow] potiuk commented on pull request #23570: The output of commands of Breeze are only generated when they change

2022-05-09 Thread GitBox
potiuk commented on PR #23570: URL: https://github.com/apache/airflow/pull/23570#issuecomment-1120895217 Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk merged pull request #23570: The output of commands of Breeze are only generated when they change

2022-05-09 Thread GitBox
potiuk merged PR #23570: URL: https://github.com/apache/airflow/pull/23570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #23570: The output of commands of Breeze are only generated when they change

2022-05-09 Thread GitBox
potiuk commented on PR #23570: URL: https://github.com/apache/airflow/pull/23570#issuecomment-1120895058 Jsut random test refused to start. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk closed issue #22908: Breeze: Add nicer error for BREEZE.rst update static check

2022-05-09 Thread GitBox
potiuk closed issue #22908: Breeze: Add nicer error for BREEZE.rst update static check URL: https://github.com/apache/airflow/issues/22908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk opened a new pull request, #23582: Fixed option name in Breeze description

2022-05-09 Thread GitBox
potiuk opened a new pull request, #23582: URL: https://github.com/apache/airflow/pull/23582 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] github-actions[bot] commented on pull request #23579: Update dags.rst

2022-05-09 Thread GitBox
github-actions[bot] commented on PR #23579: URL: https://github.com/apache/airflow/pull/23579#issuecomment-1120690259 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full

[GitHub] [airflow] ephraimbuddy commented on issue #23227: Ability to clear a specific DAG Run's task instances via REST APIs

2022-05-09 Thread GitBox
ephraimbuddy commented on issue #23227: URL: https://github.com/apache/airflow/issues/23227#issuecomment-1120747018 Another PR addressing the issue: https://github.com/apache/airflow/pull/23451 cc: @Bowrna -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] tirkarthi commented on issue #23580: `task_id` with `.` e.g. `hello.world` is not rendered in grid view

2022-05-09 Thread GitBox
tirkarthi commented on issue #23580: URL: https://github.com/apache/airflow/issues/23580#issuecomment-1120747748 cc: @bbovenzi I guess this has something to do with `task_id` having dot used to render in splitting task_id by dot :

[GitHub] [airflow] dgdelahera commented on pull request #23450: Add await all containers option to Kubernetes Pod Operator

2022-05-09 Thread GitBox
dgdelahera commented on PR #23450: URL: https://github.com/apache/airflow/pull/23450#issuecomment-1120727593 My first approach was to check the `base` container. I think it is the simplest one, but I wasn't sure it that approach was generic enough (sidecars finishing before the workload).

[GitHub] [airflow] ephraimbuddy commented on issue #23227: Ability to clear a specific DAG Run's task instances via REST APIs

2022-05-09 Thread GitBox
ephraimbuddy commented on issue #23227: URL: https://github.com/apache/airflow/issues/23227#issuecomment-1120740197 @bbovenzi @Bowrna I'm leaning towards updating the existing clearTaskInstances endpoint to optionally accept dag_run_id. This is a duplication of the existing endpoint

[GitHub] [airflow] johannesjung commented on issue #22397: Upgrade to FlaskAppBuilder 4.0.*

2022-05-09 Thread GitBox
johannesjung commented on issue #22397: URL: https://github.com/apache/airflow/issues/22397#issuecomment-1120937990 Hi @potiuk, thank you for pushing towards upgrading Flask-AppBuilder. I see that you had started to work on this issue in https://github.com/apache/airflow/pull/22587, but

[GitHub] [airflow] potiuk commented on a diff in pull request #23468: Rename cluster_policy to task_policy

2022-05-09 Thread GitBox
potiuk commented on code in PR #23468: URL: https://github.com/apache/airflow/pull/23468#discussion_r867875723 ## docs/apache-airflow/concepts/cluster-policies.rst: ## @@ -57,12 +57,17 @@ This policy checks if each DAG has at least one tag defined: Task policies -

[GitHub] [airflow] bbovenzi commented on issue #23580: `task_id` with `.` e.g. `hello.world` is not rendered in grid view

2022-05-09 Thread GitBox
bbovenzi commented on issue #23580: URL: https://github.com/apache/airflow/issues/23580#issuecomment-1120983270 Oh good catch. We can parse out the names differently to avoid this problem -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] lozuwa opened a new issue, #23584: Airflow 2.2.4 doesn't add new DAGs to the serialized_dag table, thus webserver doesn't show the new DAGs

2022-05-09 Thread GitBox
lozuwa opened a new issue, #23584: URL: https://github.com/apache/airflow/issues/23584 ### Apache Airflow version 2.2.4 ### What happened We have upgraded from Airflow 1.10.5 to Airflow 2.2.4. Upgrade path was 1.10.5 -> 1.10.15 -> 2.1.0 -> 2.2.4 Everything seems

[GitHub] [airflow] boring-cyborg[bot] commented on issue #23584: Airflow 2.2.4 doesn't add new DAGs to the serialized_dag table, thus webserver doesn't show the new DAGs

2022-05-09 Thread GitBox
boring-cyborg[bot] commented on issue #23584: URL: https://github.com/apache/airflow/issues/23584#issuecomment-1121022127 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[airflow] branch main updated: Add support for queued state in DagRun update endpoint. (#23481)

2022-05-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4485393562 Add support for queued state in DagRun

[GitHub] [airflow] potiuk commented on a diff in pull request #23481: Add support for queued state in DagRun update endpoint.

2022-05-09 Thread GitBox
potiuk commented on code in PR #23481: URL: https://github.com/apache/airflow/pull/23481#discussion_r867954118 ## airflow/api_connexion/openapi/v1.yaml: ## @@ -2442,6 +2442,7 @@ components: enum: - success - failed +- queued

[GitHub] [airflow] potiuk merged pull request #23481: Add support for queued state in DagRun update endpoint.

2022-05-09 Thread GitBox
potiuk merged PR #23481: URL: https://github.com/apache/airflow/pull/23481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk closed issue #23420: Add a queue DAG run endpoint to REST API

2022-05-09 Thread GitBox
potiuk closed issue #23420: Add a queue DAG run endpoint to REST API URL: https://github.com/apache/airflow/issues/23420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] snjypl commented on issue #21087: KubernetesJobWatcher failing on HTTP 410 errors, jobs stuck in scheduled state

2022-05-09 Thread GitBox
snjypl commented on issue #21087: URL: https://github.com/apache/airflow/issues/21087#issuecomment-1121052643 @ecerulm as i understand there are two aspects to this issue: **1.) preventing 410 error from happening:** To prevent it we need to use the BOOKMARK feature. it

[GitHub] [airflow] potiuk merged pull request #23486: Fix scheduler crash when expanding with mapped task that returned none

2022-05-09 Thread GitBox
potiuk merged PR #23486: URL: https://github.com/apache/airflow/pull/23486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (4485393562 -> 7813f996ab)

2022-05-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 4485393562 Add support for queued state in DagRun update endpoint. (#23481) add 7813f996ab Fix scheduler crash

[GitHub] [airflow] potiuk commented on pull request #19702: Switch Markdown engine to markdown-it-py

2022-05-09 Thread GitBox
potiuk commented on PR #19702: URL: https://github.com/apache/airflow/pull/19702#issuecomment-1121085277 @torbjornvatn -> can you rebase this one on top of latest main in case it fails? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] potiuk commented on issue #15150: "duplicate key value violates unique constraint "dag_run_dag_id_execution_date_key" when triggering a DAG

2022-05-09 Thread GitBox
potiuk commented on issue #15150: URL: https://github.com/apache/airflow/issues/15150#issuecomment-1121125655 > Seems strange to have that unique constraint. If two DAG runs have different run_ids, why not let them run at the same time? Because historically execution date was the key

[GitHub] [airflow] josh-fell commented on a diff in pull request #23559: Fix `PythonVirtualenvOperator` templated_fields

2022-05-09 Thread GitBox
josh-fell commented on code in PR #23559: URL: https://github.com/apache/airflow/pull/23559#discussion_r868039292 ## airflow/operators/python.py: ## @@ -327,7 +327,13 @@ class PythonVirtualenvOperator(PythonOperator): processing templated fields, for examples

[GitHub] [airflow] ianbuss commented on pull request #23535: Fix regression in ignoring symlinks

2022-05-09 Thread GitBox
ianbuss commented on PR #23535: URL: https://github.com/apache/airflow/pull/23535#issuecomment-1121189758 @tirkarthi @NickYadance could you take a look at this PR to make sure it covers the issue you're facing in #23532 . @potiuk added some basic infinite recursion detection logic

[GitHub] [airflow] thinhnd2104 commented on issue #23469: Airflow ecs fail to fetch logs from cloudwatch

2022-05-09 Thread GitBox
thinhnd2104 commented on issue #23469: URL: https://github.com/apache/airflow/issues/23469#issuecomment-1120940781 @potiuk yes. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] thinhnd2104 closed issue #23469: Airflow ecs fail to fetch logs from cloudwatch

2022-05-09 Thread GitBox
thinhnd2104 closed issue #23469: Airflow ecs fail to fetch logs from cloudwatch URL: https://github.com/apache/airflow/issues/23469 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[airflow] branch constraints-main updated: Updating constraints. Build id:2293499725

2022-05-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 98571ecb66 Updating

[GitHub] [airflow] johannesjung commented on issue #22397: Upgrade to FlaskAppBuilder 4.0.*

2022-05-09 Thread GitBox
johannesjung commented on issue #22397: URL: https://github.com/apache/airflow/issues/22397#issuecomment-1120960388 Thanks for the quick reply. It is not my field of expertise, so I would be happy if someone else gives it a try. It would be great to get rid of the older version of

[GitHub] [airflow] potiuk commented on issue #23085: Breeze: Running K8S tests with Kind with Breeze

2022-05-09 Thread GitBox
potiuk commented on issue #23085: URL: https://github.com/apache/airflow/issues/23085#issuecomment-1121073063 Hey @edithturn - how is it going :) ? I am reaching the end of my other changes and might pick it up too. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow-site] potiuk merged pull request #589: Refresh the documentation with K8S library clarification

2022-05-09 Thread GitBox
potiuk merged PR #589: URL: https://github.com/apache/airflow-site/pull/589 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #23090: Use inclusive words in apache airflow project

2022-05-09 Thread GitBox
potiuk commented on PR #23090: URL: https://github.com/apache/airflow/pull/23090#issuecomment-1121127906 Can you please rebase @edithturn - we had some refactoring/changes and this one is now conflicting. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] github-actions[bot] commented on pull request #23559: Fix `PythonVirtualenvOperator` templated_fields

2022-05-09 Thread GitBox
github-actions[bot] commented on PR #23559: URL: https://github.com/apache/airflow/pull/23559#issuecomment-1121132712 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the

[GitHub] [airflow] github-actions[bot] commented on pull request #23559: Fix `PythonVirtualenvOperator` templated_fields

2022-05-09 Thread GitBox
github-actions[bot] commented on PR #23559: URL: https://github.com/apache/airflow/pull/23559#issuecomment-1121132715 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the

[GitHub] [airflow] Bowrna commented on a diff in pull request #23516: clear specific dag run TI

2022-05-09 Thread GitBox
Bowrna commented on code in PR #23516: URL: https://github.com/apache/airflow/pull/23516#discussion_r868046251 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -450,6 +451,99 @@ def post_clear_task_instances(*, dag_id: str, session: Session = NEW_SESSION) ->

[GitHub] [airflow] potiuk commented on issue #23497: Tasks stuck indefinitely when following container logs

2022-05-09 Thread GitBox
potiuk commented on issue #23497: URL: https://github.com/apache/airflow/issues/23497#issuecomment-1121187437 Ah ok. I see. Would you like to propose a PR to fix it ? You seem to understand very well both the root cause, you can apparently reproduce it and you seem to understand well how

[GitHub] [airflow] potiuk commented on issue #22397: Upgrade to FlaskAppBuilder 4.0.*

2022-05-09 Thread GitBox
potiuk commented on issue #22397: URL: https://github.com/apache/airflow/issues/22397#issuecomment-1120948926 This was just a draft to try it and see if it works, It had a number of problems in just "dependencies". It means that it's not "straightforward" and someone will need to make a

[GitHub] [airflow] potiuk commented on issue #23453: airflow db upgrade: FOREIGN KEY constraint failed

2022-05-09 Thread GitBox
potiuk commented on issue #23453: URL: https://github.com/apache/airflow/issues/23453#issuecomment-1121061522 I would not worry about sqlite. By definition it's only for development and testing and it's ok to remove any sqlite database and recreate it from the scratch. Should we close this

[GitHub] [airflow] potiuk closed pull request #19702: Switch Markdown engine to markdown-it-py

2022-05-09 Thread GitBox
potiuk closed pull request #19702: Switch Markdown engine to markdown-it-py URL: https://github.com/apache/airflow/pull/19702 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] torbjornvatn opened a new pull request, #19702: Switch Markdown engine to markdown-it-py

2022-05-09 Thread GitBox
torbjornvatn opened a new pull request, #19702: URL: https://github.com/apache/airflow/pull/19702 As mentioned in #16435 will switching to markdown-it-py give much more advanced Markdown rendering and make it behave almost like to Github Flavored Markdown closes: #16435 related:

[GitHub] [airflow] potiuk commented on pull request #19702: Switch Markdown engine to markdown-it-py

2022-05-09 Thread GitBox
potiuk commented on PR #19702: URL: https://github.com/apache/airflow/pull/19702#issuecomment-1121084608 Reopened it to rebuild - but I am afraid there is a change that might fail the K8S tests. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] josh-fell commented on pull request #22938: Enable dbt Cloud provider to interact with single tenant instances

2022-05-09 Thread GitBox
josh-fell commented on PR #22938: URL: https://github.com/apache/airflow/pull/22938#issuecomment-1121135108 @epapineau How would users typically connect to these single-tenant instances? Does one API key have access to multiple, single-tenant instances or should one API only have access to

[GitHub] [airflow] github-actions[bot] commented on pull request #23582: Fixed option name in Breeze description

2022-05-09 Thread GitBox
github-actions[bot] commented on PR #23582: URL: https://github.com/apache/airflow/pull/23582#issuecomment-1120910273 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full

[GitHub] [airflow] tirkarthi commented on issue #23447: External DAG processor not working

2022-05-09 Thread GitBox
tirkarthi commented on issue #23447: URL: https://github.com/apache/airflow/issues/23447#issuecomment-1120956230 I guess this can be closed now with https://github.com/apache/airflow/pull/23575 merged . -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on issue #23447: External DAG processor not working

2022-05-09 Thread GitBox
potiuk commented on issue #23447: URL: https://github.com/apache/airflow/issues/23447#issuecomment-1121014156 Indeed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk closed issue #23447: External DAG processor not working

2022-05-09 Thread GitBox
potiuk closed issue #23447: External DAG processor not working URL: https://github.com/apache/airflow/issues/23447 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk commented on pull request #23562: Improve caching for multi-platform images.

2022-05-09 Thread GitBox
potiuk commented on PR #23562: URL: https://github.com/apache/airflow/pull/23562#issuecomment-1121057147 I'd love get that one merged to get it in the hands of Breeze users. I will have to refresh cache before merge. But I think it will be really nice improvement - finally :) -- This is

[GitHub] [airflow] potiuk commented on issue #23495: Defining custom jobs/hooks in the Helm chart would be a great feature

2022-05-09 Thread GitBox
potiuk commented on issue #23495: URL: https://github.com/apache/airflow/issues/23495#issuecomment-1121077872 Can you do that by extending the helm-chart? I believe Helm charts can be extended by making the subcharts , so you can write your own chart with Airflow chart as a Subchart and

[GitHub] [airflow] potiuk commented on a diff in pull request #23506: Added postgres 14 to support versions(including breeze)

2022-05-09 Thread GitBox
potiuk commented on code in PR #23506: URL: https://github.com/apache/airflow/pull/23506#discussion_r868022398 ## scripts/ci/libraries/_initialization.sh: ## @@ -125,7 +125,7 @@ function initialization::initialize_base_variables() { export

[GitHub] [airflow] rotemseekingalpha commented on issue #23588: After upgrade from Airflow 2.2.4, grid disappears for some DAGs

2022-05-09 Thread GitBox
rotemseekingalpha commented on issue #23588: URL: https://github.com/apache/airflow/issues/23588#issuecomment-1121123177 [Video](https://youtu.be/BD91yCd6r68) that shows the problem -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on issue #9967: task instance is scheduled repeatedly while dag file is pretty large

2022-05-09 Thread GitBox
potiuk commented on issue #9967: URL: https://github.com/apache/airflow/issues/9967#issuecomment-1121122511 Cool. Thanks for sharing your case @flaviomax . I add this use case to my portfolio of "potential problems to suggest our users to look for :)". Actually that might be a cool idea to

[GitHub] [airflow] ecerulm commented on issue #21087: KubernetesJobWatcher failing on HTTP 410 errors, jobs stuck in scheduled state

2022-05-09 Thread GitBox
ecerulm commented on issue #21087: URL: https://github.com/apache/airflow/issues/21087#issuecomment-1121165082 @snjypl , let me see if I can address you comments (in no particular order, sorry) ### allow_watch_bookmark=True does not prevent 410 > it is because of this bug in

[airflow] branch main updated (7c7b001a63 -> 1220c1a7a9)

2022-05-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 7c7b001a63 tHe output of commands of Breeze are only generated when they change (#23570) add 1220c1a7a9 Fixed

[GitHub] [airflow] potiuk merged pull request #23582: Fixed option name in Breeze description

2022-05-09 Thread GitBox
potiuk merged PR #23582: URL: https://github.com/apache/airflow/pull/23582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] bbovenzi commented on issue #23417: Move grid_data endpoint to REST API

2022-05-09 Thread GitBox
bbovenzi commented on issue #23417: URL: https://github.com/apache/airflow/issues/23417#issuecomment-1120987663 oh yes, we renamed it to `grid_data`. I updated the link too. We probably don't need to return the `dag_runs` part of the response and instead use the list dag runs

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #23516: clear specific dag run TI

2022-05-09 Thread GitBox
ephraimbuddy commented on code in PR #23516: URL: https://github.com/apache/airflow/pull/23516#discussion_r867938334 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -450,6 +451,99 @@ def post_clear_task_instances(*, dag_id: str, session: Session =

[GitHub] [airflow] ashb commented on a diff in pull request #23506: Added postgres 14 to support versions(including breeze)

2022-05-09 Thread GitBox
ashb commented on code in PR #23506: URL: https://github.com/apache/airflow/pull/23506#discussion_r868026504 ## scripts/ci/libraries/_initialization.sh: ## @@ -125,7 +125,7 @@ function initialization::initialize_base_variables() { export CURRENT_PYTHON_MAJOR_MINOR_VERSIONS

[GitHub] [airflow] jpipas commented on issue #23588: After upgrade from Airflow 2.2.4, grid disappears for some DAGs

2022-05-09 Thread GitBox
jpipas commented on issue #23588: URL: https://github.com/apache/airflow/issues/23588#issuecomment-1121172291 Getting the same thing - It appears to be an issue with the getTask API (there's several javascript errors in console). Most notably the /tasks endpoint for a given dag.

[GitHub] [airflow] ldacey commented on issue #23472: Airflow hangs on "Found x duplicates in table task_fail. Will attempt to move them"

2022-05-09 Thread GitBox
ldacey commented on issue #23472: URL: https://github.com/apache/airflow/issues/23472#issuecomment-1121184528 Sorry for the late reply, I am using PostgreSQL. I assume that I just became impatient and canceled the db upgrade command prematurely. I am not sure why my task_fail table had

[GitHub] [airflow] potiuk commented on issue #23469: Airflow ecs fail to fetch logs from cloudwatch

2022-05-09 Thread GitBox
potiuk commented on issue #23469: URL: https://github.com/apache/airflow/issues/23469#issuecomment-1120939085 @thinhnd2104 - do you think the issue is closed by #23470 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] ecerulm commented on issue #21087: KubernetesJobWatcher failing on HTTP 410 errors, jobs stuck in scheduled state

2022-05-09 Thread GitBox
ecerulm commented on issue #21087: URL: https://github.com/apache/airflow/issues/21087#issuecomment-1120994210 @snjypl, does not `allow_watch_bookmarks` prevent a "410 GONE", and does not prevent the current airflow code path to enter an infinite loop because of it. The current

[GitHub] [airflow] potiuk commented on issue #22397: Upgrade to FlaskAppBuilder 4.0.*

2022-05-09 Thread GitBox
potiuk commented on issue #22397: URL: https://github.com/apache/airflow/issues/22397#issuecomment-1121015252 > Thanks for the quick reply. It is not my field of expertise, so I would be happy if someone else gives it a try. It would be great to get rid of the older version of dependencies

[GitHub] [airflow] wojsamjan opened a new pull request, #23583: Cloud SQL assets & system tests migration (AIP-47)

2022-05-09 Thread GitBox
wojsamjan opened a new pull request, #23583: URL: https://github.com/apache/airflow/pull/23583 Add links to assets for Cloud SQL & migrate system tests (AIP-47) Co-authored-by: Wojciech Januszek [janus...@google.com](mailto:janus...@google.com) Co-authored-by: Lukasz

[GitHub] [airflow] potiuk opened a new issue, #23585: Synchronizing Airlfow releases, API versions and clients

2022-05-09 Thread GitBox
potiuk opened a new issue, #23585: URL: https://github.com/apache/airflow/issues/23585 ### Body The current version of API https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html shows version 1.0.0. I think this is terribly wrong and we shoudl synchronize

[GitHub] [airflow] snjypl commented on issue #21087: KubernetesJobWatcher failing on HTTP 410 errors, jobs stuck in scheduled state

2022-05-09 Thread GitBox
snjypl commented on issue #21087: URL: https://github.com/apache/airflow/issues/21087#issuecomment-1121062446 > So again there is never a guarantee that the last resource version that you got is going to be valid the next time that you start the watch, no matter how well you

[GitHub] [airflow-site] potiuk commented on pull request #589: Refresh the documentation with K8S library clarification

2022-05-09 Thread GitBox
potiuk commented on PR #589: URL: https://github.com/apache/airflow-site/pull/589#issuecomment-1121071852 Anyone :) ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] snjypl commented on issue #21087: KubernetesJobWatcher failing on HTTP 410 errors, jobs stuck in scheduled state

2022-05-09 Thread GitBox
snjypl commented on issue #21087: URL: https://github.com/apache/airflow/issues/21087#issuecomment-1121079448 the code you shared is not really a good way to demonstrate `BOOKMARK`. but still: > for i in watcher.stream(v1.list_namespaced_pod, namespace, resource_version=1,

[GitHub] [airflow] potiuk closed issue #23497: Tasks stuck indefinitely when following container logs

2022-05-09 Thread GitBox
potiuk closed issue #23497: Tasks stuck indefinitely when following container logs URL: https://github.com/apache/airflow/issues/23497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] potiuk commented on issue #23497: Tasks stuck indefinitely when following container logs

2022-05-09 Thread GitBox
potiuk commented on issue #23497: URL: https://github.com/apache/airflow/issues/23497#issuecomment-1121089809 Can you please try it with Airflow 2.3.0 @schattian - one of the changes in 2.3.0 (And corresponding cncf provider 4.0+ was to migrate to newer kubernetes libraries that are

[GitHub] [airflow] potiuk commented on issue #17176: TriggerDagRunOperator triggering subdags out of order

2022-05-09 Thread GitBox
potiuk commented on issue #17176: URL: https://github.com/apache/airflow/issues/17176#issuecomment-1121095023 I think not. The long term solution is to switch to Task Group. Subdags are deprecated and are going to be removed in Airflow 3 anyway, so the best we can advice is to switch to

[GitHub] [airflow] potiuk closed issue #17176: TriggerDagRunOperator triggering subdags out of order

2022-05-09 Thread GitBox
potiuk closed issue #17176: TriggerDagRunOperator triggering subdags out of order URL: https://github.com/apache/airflow/issues/17176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] rotemseekingalpha opened a new issue, #23588: After upgrade from Airflow 2.2.4, grid disappears for some DAGs

2022-05-09 Thread GitBox
rotemseekingalpha opened a new issue, #23588: URL: https://github.com/apache/airflow/issues/23588 ### Apache Airflow version 2.3.0 (latest released) ### What happened After the upgrade from 2.2.4 to 2.3.0, some DAGs grid data seems missing and it renders the UI blank

[GitHub] [airflow] boring-cyborg[bot] commented on issue #23588: After upgrade from Airflow 2.2.4, grid disappears for some DAGs

2022-05-09 Thread GitBox
boring-cyborg[bot] commented on issue #23588: URL: https://github.com/apache/airflow/issues/23588#issuecomment-1121114782 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on a diff in pull request #23506: Added postgres 14 to support versions(including breeze)

2022-05-09 Thread GitBox
potiuk commented on code in PR #23506: URL: https://github.com/apache/airflow/pull/23506#discussion_r868023074 ## scripts/ci/libraries/_initialization.sh: ## @@ -125,7 +125,7 @@ function initialization::initialize_base_variables() { export

[GitHub] [airflow] ashb commented on a diff in pull request #23535: Fix regression in ignoring symlinks

2022-05-09 Thread GitBox
ashb commented on code in PR #23535: URL: https://github.com/apache/airflow/pull/23535#discussion_r868030054 ## tests/utils/test_file.py: ## @@ -78,6 +78,29 @@ def test_open_maybe_zipped_archive(self): class TestListPyFilesPath(unittest.TestCase): +def setUp(self):

[GitHub] [airflow] schattian commented on issue #23497: Tasks stuck indefinitely when following container logs

2022-05-09 Thread GitBox
schattian commented on issue #23497: URL: https://github.com/apache/airflow/issues/23497#issuecomment-1121121785 @potiuk hm I don't see how that could help here. The code here continuous to be blocking execute:

[GitHub] [airflow] potiuk commented on issue #23513: Flower do not honor FLOWER_TASK_RUNTIME_METRIC_BUCKETS env parameter

2022-05-09 Thread GitBox
potiuk commented on issue #23513: URL: https://github.com/apache/airflow/issues/23513#issuecomment-1121141870 If you can see the environment properly set in the flower container, there is no reason why it should not work - please double-check (exec into your container and check if it is

[GitHub] [airflow] bbovenzi commented on issue #23227: Ability to clear a specific DAG Run's task instances via REST APIs

2022-05-09 Thread GitBox
bbovenzi commented on issue #23227: URL: https://github.com/apache/airflow/issues/23227#issuecomment-1121141332 @ephraimbuddy That's fine with me to just add `dag_run_id` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] potiuk commented on pull request #23036: Add missing container_timeout argument

2022-05-09 Thread GitBox
potiuk commented on PR #23036: URL: https://github.com/apache/airflow/pull/23036#issuecomment-1121146826 > I might be completely missing it, but where is this parameter being used? Seems like this change creates an instance attribute of `container_timeout` and remains unused. It

  1   2   3   >