[GitHub] [airflow] PatrykKlimowicz commented on issue #25177: Airflow ElasticSearch provider issue

2022-07-21 Thread GitBox
PatrykKlimowicz commented on issue #25177: URL: https://github.com/apache/airflow/issues/25177#issuecomment-1192248250 @potiuk I'll try to test 😄 Will be back with some feedback soon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [airflow-site] Bowrna commented on pull request #623: replacing site.sh with python

2022-07-21 Thread GitBox
Bowrna commented on PR #623: URL: https://github.com/apache/airflow-site/pull/623#issuecomment-1192201127 > @Bowrna see: https://stackoverflow.com/a/12498485 This generates a relative path for use in a container. Could you tell me why we need to convert the absolute path into relative

[GitHub] [airflow] uranusjr commented on pull request #25215: Fix ExternalTaskSensor not working with dynamic task

2022-07-21 Thread GitBox
uranusjr commented on PR #25215: URL: https://github.com/apache/airflow/pull/25215#issuecomment-1192176606 Could you also do the same for `ExternalTaskMarker`? It’s in the same file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [airflow] uranusjr commented on issue #25165: Dynamic Tasks inside of TaskGroup do not have group_id prepended to task_id

2022-07-21 Thread GitBox
uranusjr commented on issue #25165: URL: https://github.com/apache/airflow/issues/25165#issuecomment-1192167588 > I think the "root" fix is to prepend the group_id to MappedOperator.task_id, like what BaseOperator does This logic already exists; a `@task`-decorated task calls `get_un

[GitHub] [airflow] uranusjr commented on pull request #25218: Upgrade to Pip 22.2

2022-07-21 Thread GitBox
uranusjr commented on PR #25218: URL: https://github.com/apache/airflow/pull/25218#issuecomment-1192147059 pip doesn’t do semver because there is not any reasonable point it could ever bump major, and even when we need to make incompatible changes, we need drag all the users to the new vers

[GitHub] [airflow] eladkal commented on pull request #23654: Update download_file method in s3_hook to accept files and directories

2022-07-21 Thread GitBox
eladkal commented on PR #23654: URL: https://github.com/apache/airflow/pull/23654#issuecomment-1192117375 @kalyani-subbiah are you still working on this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] amoGLingle commented on issue #24909: Airflow Scheduler Deadlock - Transaction not rolled back on Exception?

2022-07-21 Thread GitBox
amoGLingle commented on issue #24909: URL: https://github.com/apache/airflow/issues/24909#issuecomment-1192115740 As an experiment, we're turning off one of the Schedulers - no HA - to see if we still get a deadlock. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] boring-cyborg[bot] commented on issue #25226: worker service selector selects pods launched by Kubernetes executor with default charts pod template

2022-07-21 Thread GitBox
boring-cyborg[bot] commented on issue #25226: URL: https://github.com/apache/airflow/issues/25226#issuecomment-1192077021 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] pewpewp3w opened a new issue, #25226: worker service selector selects pods launched by Kubernetes executor with default charts pod template

2022-07-21 Thread GitBox
pewpewp3w opened a new issue, #25226: URL: https://github.com/apache/airflow/issues/25226 ### Official Helm Chart version 1.6.0 (latest released) ### Apache Airflow version 2.3.3 (latest released) ### Kubernetes Version does not matter ### Helm Chart

[GitHub] [airflow] github-actions[bot] commented on issue #18628: The cleared dagrun and tasks are set failed by scheduler

2022-07-21 Thread GitBox
github-actions[bot] commented on issue #18628: URL: https://github.com/apache/airflow/issues/18628#issuecomment-1192050435 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] github-actions[bot] closed issue #18628: The cleared dagrun and tasks are set failed by scheduler

2022-07-21 Thread GitBox
github-actions[bot] closed issue #18628: The cleared dagrun and tasks are set failed by scheduler URL: https://github.com/apache/airflow/issues/18628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] github-actions[bot] closed pull request #22806: Helmchart: Allow digest version to be used

2022-07-21 Thread GitBox
github-actions[bot] closed pull request #22806: Helmchart: Allow digest version to be used URL: https://github.com/apache/airflow/pull/22806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [airflow] github-actions[bot] closed issue #24400: Airflow db upgrade fails to insert rows in log_template table

2022-07-21 Thread GitBox
github-actions[bot] closed issue #24400: Airflow db upgrade fails to insert rows in log_template table URL: https://github.com/apache/airflow/issues/24400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [airflow] github-actions[bot] commented on issue #24400: Airflow db upgrade fails to insert rows in log_template table

2022-07-21 Thread GitBox
github-actions[bot] commented on issue #24400: URL: https://github.com/apache/airflow/issues/24400#issuecomment-1192050381 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

[airflow] branch main updated: Upgrade to Pip 22.2 (#25218)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1cb16d5588 Upgrade to Pip 22.2 (#25218) 1cb16d5588

[GitHub] [airflow] potiuk merged pull request #25218: Upgrade to Pip 22.2

2022-07-21 Thread GitBox
potiuk merged PR #25218: URL: https://github.com/apache/airflow/pull/25218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #25218: Upgrade to Pip 22.2

2022-07-21 Thread GitBox
potiuk commented on PR #25218: URL: https://github.com/apache/airflow/pull/25218#issuecomment-1192024482 > Interesting they don't include `.0`. Yep. `pypi` does not use SemVer. And while you can use .0 in the first version, th form of X.Y is generally ok for all the projects - not on

[GitHub] [airflow] potiuk commented on pull request #25224: Convert example_ecs_fargate.py to an AIP-47 system test

2022-07-21 Thread GitBox
potiuk commented on PR #25224: URL: https://github.com/apache/airflow/pull/25224#issuecomment-1192010510 > Huh. Where did those come from 👀 > > Looks like the same tests are failing in main, so it's something that was changed in the last couple of days. I'm heading out for a mini

[GitHub] [airflow] erdos2n commented on issue #25165: Dynamic Tasks inside of TaskGroup do not have group_id prepended to task_id

2022-07-21 Thread GitBox
erdos2n commented on issue #25165: URL: https://github.com/apache/airflow/issues/25165#issuecomment-1191975901 I don't know if I can assign myself... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] bbovenzi commented on issue #25165: Dynamic Tasks inside of TaskGroup do not have group_id prepended to task_id

2022-07-21 Thread GitBox
bbovenzi commented on issue #25165: URL: https://github.com/apache/airflow/issues/25165#issuecomment-1191967441 @erdos2n go for it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [airflow] ferruzzi commented on pull request #25224: Convert example_ecs_fargate.py to an AIP-47 system test

2022-07-21 Thread GitBox
ferruzzi commented on PR #25224: URL: https://github.com/apache/airflow/pull/25224#issuecomment-1191963364 Huh. Where did those come from :eyes: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] erdos2n commented on issue #25165: Dynamic Tasks inside of TaskGroup do not have group_id prepended to task_id

2022-07-21 Thread GitBox
erdos2n commented on issue #25165: URL: https://github.com/apache/airflow/issues/25165#issuecomment-1191963111 Can I take this @bbovenzi ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [airflow] potiuk commented on a diff in pull request #25185: 25092: MsSqlHook: fix for get_uri, adding pymssql driver to scheme

2022-07-21 Thread GitBox
potiuk commented on code in PR #25185: URL: https://github.com/apache/airflow/pull/25185#discussion_r927104026 ## airflow/providers/microsoft/mssql/hooks/mssql.py: ## @@ -51,6 +51,13 @@ def get_conn( ) return conn +def get_uri(self) -> str: +from

[GitHub] [airflow] potiuk commented on a diff in pull request #25185: 25092: MsSqlHook: fix for get_uri, adding pymssql driver to scheme

2022-07-21 Thread GitBox
potiuk commented on code in PR #25185: URL: https://github.com/apache/airflow/pull/25185#discussion_r927103463 ## airflow/providers/microsoft/mssql/hooks/mssql.py: ## @@ -51,6 +51,13 @@ def get_conn( ) return conn +def get_uri(self) -> str: +from

[airflow] branch main updated: Stop SLA callbacks gazumping other callbacks and DOS'ing the DagProcessorManager queue (#25147)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 17ec6dbcfe Stop SLA callbacks gazumping other callb

[GitHub] [airflow] potiuk merged pull request #25147: Stop SLA callbacks gazumping other callbacks and DOS'ing the DagProcessorManager queue

2022-07-21 Thread GitBox
potiuk merged PR #25147: URL: https://github.com/apache/airflow/pull/25147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #25147: Stop SLA callbacks gazumping other callbacks and DOS'ing the DagProcessorManager queue

2022-07-21 Thread GitBox
potiuk commented on PR #25147: URL: https://github.com/apache/airflow/pull/25147#issuecomment-1191932877 Merging now - unless someone will have objections - it's simple enough change to be reverted/improved on in case. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [airflow] potiuk commented on pull request #25215: Fix ExternalTaskSensor not working with dynamic task

2022-07-21 Thread GitBox
potiuk commented on PR #25215: URL: https://github.com/apache/airflow/pull/25215#issuecomment-1191932073 @uranusjr ? does it make sense? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [airflow] potiuk closed issue #25211: Airflow scheduler spawn zombie process

2022-07-21 Thread GitBox
potiuk closed issue #25211: Airflow scheduler spawn zombie process URL: https://github.com/apache/airflow/issues/25211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [airflow] potiuk commented on pull request #25183: Add auth_type to LivyHook

2022-07-21 Thread GitBox
potiuk commented on PR #25183: URL: https://github.com/apache/airflow/pull/25183#issuecomment-1191927935 You should add unit tests covering it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #25207: Create custom DAG classes with custom `clear` etc.

2022-07-21 Thread GitBox
potiuk commented on issue #25207: URL: https://github.com/apache/airflow/issues/25207#issuecomment-1191926593 Also to add - the very important part of this is security. We are working on improving the security of Airflow and we are gradually fixing all the potential places where - especiall

[GitHub] [airflow] ferruzzi opened a new pull request, #25224: Convert example_ecs_fargate.py to an AIP-47 system test

2022-07-21 Thread GitBox
ferruzzi opened a new pull request, #25224: URL: https://github.com/apache/airflow/pull/25224 Follows the template set forward in https://github.com/apache/airflow/pull/24643 Also cleans up and makes some required changes within the SystemTestContextBuilder to account for occasionall

[GitHub] [airflow] potiuk closed issue #25209: Airflow database migration version scripts are unsafe, have failed, and are guaranteed to fail in the future

2022-07-21 Thread GitBox
potiuk closed issue #25209: Airflow database migration version scripts are unsafe, have failed, and are guaranteed to fail in the future URL: https://github.com/apache/airflow/issues/25209 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on issue #25209: Airflow database migration version scripts are unsafe, have failed, and are guaranteed to fail in the future

2022-07-21 Thread GitBox
potiuk commented on issue #25209: URL: https://github.com/apache/airflow/issues/25209#issuecomment-1191915218 @vitaly-krugl - if you would like to propose some guidelines - feel free to make PR. Like everything else in Airflow., Airflow is created by > 2100 contributors and if you wou

[airflow] branch main updated: Convert RDS Export Sample DAG to System Test (AIP-47) (#25205)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f6bda38e20 Convert RDS Export Sample DAG to System

[GitHub] [airflow] potiuk merged pull request #25205: Convert RDS Export Sample DAG to System Test (AIP-47)

2022-07-21 Thread GitBox
potiuk merged PR #25205: URL: https://github.com/apache/airflow/pull/25205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] bbovenzi merged pull request #25208: Dataset details view

2022-07-21 Thread GitBox
bbovenzi merged PR #25208: URL: https://github.com/apache/airflow/pull/25208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[airflow] branch main updated (93b7a57cbc -> 210ad6408f)

2022-07-21 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 93b7a57cbc Include missing mention of `external_executor_id` in `sql_engine_collation_for_ids` docs (#25197) ad

[GitHub] [airflow] potiuk commented on pull request #25135: AIP-47 - Migrate Airbyte DAGs to new design

2022-07-21 Thread GitBox
potiuk commented on PR #25135: URL: https://github.com/apache/airflow/pull/25135#issuecomment-1191905182 @chethanuk-plutoflume - I am not sure what the problem is - but the instructions are pretty self-explanatory. Am I missing something? -- This is an automated message from the Apache G

[GitHub] [airflow] potiuk commented on pull request #25135: AIP-47 - Migrate Airbyte DAGs to new design

2022-07-21 Thread GitBox
potiuk commented on PR #25135: URL: https://github.com/apache/airflow/pull/25135#issuecomment-1191904426 Just copying the exact instructions: ``` Make sure to add those lines to docs/apache-airflow-providers-airbyte/index.rst: .. toctree:: :hidden: :caption

[GitHub] [airflow] potiuk merged pull request #25197: Include missing mention of `external_executor_id` in docstring

2022-07-21 Thread GitBox
potiuk merged PR #25197: URL: https://github.com/apache/airflow/pull/25197 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated: Include missing mention of `external_executor_id` in `sql_engine_collation_for_ids` docs (#25197)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 93b7a57cbc Include missing mention of `external_exe

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #25197: Include missing mention of `external_executor_id` in docstring

2022-07-21 Thread GitBox
boring-cyborg[bot] commented on PR #25197: URL: https://github.com/apache/airflow/pull/25197#issuecomment-1191903543 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] potiuk commented on pull request #25218: Upgrade to Pip 22.2

2022-07-21 Thread GitBox
potiuk commented on PR #25218: URL: https://github.com/apache/airflow/pull/25218#issuecomment-1191897907 :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] potiuk commented on issue #25186: should we normalize scheme and hostname of URI to be lower?

2022-07-21 Thread GitBox
potiuk commented on issue #25186: URL: https://github.com/apache/airflow/issues/25186#issuecomment-1191885294 100% on board. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] dstandish commented on issue #25186: should we normalize scheme and hostname of URI to be lower?

2022-07-21 Thread GitBox
dstandish commented on issue #25186: URL: https://github.com/apache/airflow/issues/25186#issuecomment-1191854525 I am actually thinking now that we should abandon URI as the identifier for datasets. I think part of the motivation for choosing URI was [this document in openlineage](https:/

[GitHub] [airflow] boring-cyborg[bot] commented on issue #25222: Launch Pod/job in separate namespace and support in helm chart

2022-07-21 Thread GitBox
boring-cyborg[bot] commented on issue #25222: URL: https://github.com/apache/airflow/issues/25222#issuecomment-1191837770 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] mikebell90 opened a new issue, #25222: Launch Pod/job in separate namespace and support in helm chart

2022-07-21 Thread GitBox
mikebell90 opened a new issue, #25222: URL: https://github.com/apache/airflow/issues/25222 ### Description Our engineers want to run jobs in a separate namespace from the installed release via helm chart. It looks to me like right now the rbac configuration has two choices

[airflow] branch v2-3-test updated: Temporary remove common.sql and pysftp until we release 2.3.4

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-3-test by this push: new f431503dc3 Temporary remove common.sql an

[airflow] branch main updated (47b72056c4 -> 03a81b66de)

2022-07-21 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 47b72056c4 SQSPublishOperator should allow sending messages to a FIFO Queue (#25171) add 03a81b66de Show d

[GitHub] [airflow] jedcunningham closed issue #25033: next run should show deps fulfillment e.g. 0 of 3

2022-07-21 Thread GitBox
jedcunningham closed issue #25033: next run should show deps fulfillment e.g. 0 of 3 URL: https://github.com/apache/airflow/issues/25033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] jedcunningham merged pull request #25141: Show dataset readiness for the next run

2022-07-21 Thread GitBox
jedcunningham merged PR #25141: URL: https://github.com/apache/airflow/pull/25141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] erdos2n commented on issue #23246: Add api call for changing task instance status

2022-07-21 Thread GitBox
erdos2n commented on issue #23246: URL: https://github.com/apache/airflow/issues/23246#issuecomment-1191795657 unnassigning myself, as I am unable to get the tests to pass -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [airflow] gitstart-airflow opened a new pull request, #25221: Convert TimeSensorAsync target_time to utc on call time

2022-07-21 Thread GitBox
gitstart-airflow opened a new pull request, #25221: URL: https://github.com/apache/airflow/pull/25221 This PR fixes the error raised on DateTimeTrigger when TimeSensorAsync target_time is timezone-aware. closes: #24736 related: #24736 --- **^ Add meaningful description ab

[GitHub] [airflow] denimalpaca commented on a diff in pull request #25164: Common SQLCheckOperators Various Functionality Update

2022-07-21 Thread GitBox
denimalpaca commented on code in PR #25164: URL: https://github.com/apache/airflow/pull/25164#discussion_r926969512 ## airflow/providers/common/sql/operators/sql.py: ## @@ -73,6 +79,7 @@ class SQLColumnCheckOperator(BaseSQLOperator): } } +:param batch

[GitHub] [airflow] eladkal commented on pull request #25196: Disallow any dag tags longer than 100 char

2022-07-21 Thread GitBox
eladkal commented on PR #25196: URL: https://github.com/apache/airflow/pull/25196#issuecomment-1191778707 I don't think this is right (or at least not valid to all backends) I was able to create a DAG with tag longer than 100 chars with Sqlite (didn't test the others) ![Screen Shot 202

[GitHub] [airflow] potiuk commented on a diff in pull request #25196: Disallow any dag tags longer than 100 char

2022-07-21 Thread GitBox
potiuk commented on code in PR #25196: URL: https://github.com/apache/airflow/pull/25196#discussion_r926942400 ## airflow/models/dag.py: ## @@ -347,6 +347,9 @@ def __init__( ): from airflow.utils.task_group import TaskGroup +if tags and any(len(tag) > 100

[GitHub] [airflow] potiuk commented on a diff in pull request #25164: Common SQLCheckOperators Various Functionality Update

2022-07-21 Thread GitBox
potiuk commented on code in PR #25164: URL: https://github.com/apache/airflow/pull/25164#discussion_r926935166 ## airflow/providers/common/sql/operators/sql.py: ## @@ -73,6 +79,7 @@ class SQLColumnCheckOperator(BaseSQLOperator): } } +:param batch: a S

[GitHub] [airflow] potiuk commented on a diff in pull request #25164: Common SQLCheckOperators Various Functionality Update

2022-07-21 Thread GitBox
potiuk commented on code in PR #25164: URL: https://github.com/apache/airflow/pull/25164#discussion_r926935166 ## airflow/providers/common/sql/operators/sql.py: ## @@ -73,6 +79,7 @@ class SQLColumnCheckOperator(BaseSQLOperator): } } +:param batch: a S

[airflow] 04/12: Disable provider packages building for non-main branch builds (#25056)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit bf6c0ce300cf3425b6529b39db6ec1c578987c9c Author: Jarek Potiuk AuthorDate: Tue Jul 19 14:37:44 2022 +0200 Disabl

[airflow] 09/12: Fix missing space for breeze build-image hint command (#25204)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 994a0410f44064fee083f1576b084cff59c3cb32 Author: Gabriel Machado AuthorDate: Thu Jul 21 00:23:49 2022 +0200 Fix

[airflow] 10/12: Fix asset compilation via setup.py (#25201)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 17f6105aad82b01d7f9882bb34dc9c84514831f1 Author: Jarek Potiuk AuthorDate: Wed Jul 20 23:06:16 2022 +0200 Fix as

[airflow] 02/12: Fix PR label detection in CI (#25148)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 436fe2d84877501b21e21450cf1167ec2fbd5fce Author: Jarek Potiuk AuthorDate: Tue Jul 19 12:25:17 2022 +0200 Fix PR

[airflow] 07/12: Update description of branch creation to include the new Breeze (#25155)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit f18e1657496239f8d0ae91124f2670605d97badd Author: Jarek Potiuk AuthorDate: Tue Jul 19 23:08:39 2022 +0200 Update

[airflow] 06/12: Speed up Kubernetes upgrade tests at least 2x (#25159)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 81dd5c88f0e6ad9437305b9fd21d48027fffe846 Author: Jarek Potiuk AuthorDate: Tue Jul 19 22:30:42 2022 +0200 Speed

[airflow] 11/12: Also compile assets in non-main (#25220)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 8de0ac7a753dc7409a54df9aac2ecfaa508e5af9 Author: Jarek Potiuk AuthorDate: Thu Jul 21 17:58:22 2022 +0200 Also c

[airflow] 01/12: Bump typing-extensions and mypy for ParamSpec (#25088)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit ce0caa0038d0d56d1850e2d0217e07b26bceca24 Author: Tzu-ping Chung AuthorDate: Mon Jul 18 17:20:35 2022 +0800 Bump

[airflow] 12/12: Limit astroid version to < 2.12 (#24982)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 06e4976aaf3236d06b7e6ec85a796f82718ddfcd Author: Jarek Potiuk AuthorDate: Tue Jul 12 02:32:41 2022 +0200 Limit

[airflow] 03/12: Speed up Kubernetes tests ~30% on main. (#25143)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit fa0b5b73c248a9aec8d6b407cc35840dd8ae4058 Author: Jarek Potiuk AuthorDate: Tue Jul 19 13:37:00 2022 +0200 Speed

[airflow] 05/12: Retrieve airflow branch/constraints from env variables (#25053)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 218711e601b840838a25df0b938cb05c62871292 Author: Jarek Potiuk AuthorDate: Tue Jul 19 14:38:02 2022 +0200 Retrie

[airflow] branch v2-3-test updated (acc3df56c4 -> 06e4976aaf)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git omit acc3df56c4 Also compile assets in non-main (#25220) omit d8ed1975f2 Fix asset compilation via setup.py (#25201

[GitHub] [airflow] Piatachock commented on pull request #25158: YandexCloud provider: Support new Yandex SDK features for DataProc

2022-07-21 Thread GitBox
Piatachock commented on PR #25158: URL: https://github.com/apache/airflow/pull/25158#issuecomment-1191737660 That makes sense, thanks! I guess I was not sure if pre-commit should happen on my side or on CI side. I'll see if there is other way to phrase it in README -- This is an automa

[GitHub] [airflow] jedcunningham commented on a diff in pull request #25085: Implement map() semantic

2022-07-21 Thread GitBox
jedcunningham commented on code in PR #25085: URL: https://github.com/apache/airflow/pull/25085#discussion_r926912929 ## airflow/models/taskinstance.py: ## @@ -2335,9 +2335,11 @@ def _record_task_map_for_downstreams(self, task: "Operator", value: Any, *, sess retur

[GitHub] [airflow] howardyoo commented on pull request #25219: Update statsd-exporter setup

2022-07-21 Thread GitBox
howardyoo commented on PR #25219: URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191723459 Yes, thanks for the feedback. I believe in general, metrics side of OpenTelemetry has gotten much better (I mean metrics are kind of simple by nature) - so still think the spec

[GitHub] [airflow] jedcunningham commented on pull request #25121: Add "Optimizing" chapter to dynamic-dags section

2022-07-21 Thread GitBox
jedcunningham commented on PR #25121: URL: https://github.com/apache/airflow/pull/25121#issuecomment-1191712694 > > There are other ways to DRY up a DAG than to have them all come from a single loop man_shrugging. > > Should we maybe then mention THIS instead? Maybe that is a better a

[GitHub] [airflow] uranusjr commented on pull request #25085: Implement map() semantic

2022-07-21 Thread GitBox
uranusjr commented on PR #25085: URL: https://github.com/apache/airflow/pull/25085#issuecomment-1191699342 Feel free! This one on its own shouldn’t need too much context to understand. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] denimalpaca commented on a diff in pull request #25164: Common SQLCheckOperators Various Functionality Update

2022-07-21 Thread GitBox
denimalpaca commented on code in PR #25164: URL: https://github.com/apache/airflow/pull/25164#discussion_r926885517 ## airflow/providers/common/sql/operators/sql.py: ## @@ -73,6 +79,7 @@ class SQLColumnCheckOperator(BaseSQLOperator): } } +:param batch

[airflow] branch main updated: SQSPublishOperator should allow sending messages to a FIFO Queue (#25171)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 47b72056c4 SQSPublishOperator should allow sending

[GitHub] [airflow] potiuk merged pull request #25171: SQSPublishOperator should allow sending messages to a FIFO Queue

2022-07-21 Thread GitBox
potiuk merged PR #25171: URL: https://github.com/apache/airflow/pull/25171 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk closed issue #25138: SQSPublishOperator should allow sending messages to a FIFO Queue

2022-07-21 Thread GitBox
potiuk closed issue #25138: SQSPublishOperator should allow sending messages to a FIFO Queue URL: https://github.com/apache/airflow/issues/25138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] potiuk commented on pull request #25171: SQSPublishOperator should allow sending messages to a FIFO Queue

2022-07-21 Thread GitBox
potiuk commented on PR #25171: URL: https://github.com/apache/airflow/pull/25171#issuecomment-1191694389 > @potiuk no i believe i am not able to re-run it, unless i am completely missing something Yeah. I've learned today that partial re-run from https://github.blog/2022-03-16-save-t

[GitHub] [airflow] potiuk commented on pull request #25158: YandexCloud provider: Support new Yandex SDK features for DataProc

2022-07-21 Thread GitBox
potiuk commented on PR #25158: URL: https://github.com/apache/airflow/pull/25158#issuecomment-1191692451 And feel free to add in the README that you should commit the changes generated by pre-commit (and also you could add link to https://github.com/apache/airflow/blob/main/STATIC_CODE_CHEC

[GitHub] [airflow] potiuk commented on pull request #25158: YandexCloud provider: Support new Yandex SDK features for DataProc

2022-07-21 Thread GitBox
potiuk commented on PR #25158: URL: https://github.com/apache/airflow/pull/25158#issuecomment-1191690538 It's actually explained at the bottom of the error you see in static checks and here are the docs: https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#pre-commit-hooks -

[GitHub] [airflow] potiuk commented on pull request #25158: YandexCloud provider: Support new Yandex SDK features for DataProc

2022-07-21 Thread GitBox
potiuk commented on PR #25158: URL: https://github.com/apache/airflow/pull/25158#issuecomment-1191689182 You should simply run pre-commit and commit what has been generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup

2022-07-21 Thread GitBox
potiuk commented on PR #25219: URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191679294 BTW. Some of our helm tests need fixing :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk commented on a diff in pull request #25164: Common SQLCheckOperators Various Functionality Update

2022-07-21 Thread GitBox
potiuk commented on code in PR #25164: URL: https://github.com/apache/airflow/pull/25164#discussion_r926867814 ## airflow/providers/common/sql/operators/sql.py: ## @@ -73,6 +79,7 @@ class SQLColumnCheckOperator(BaseSQLOperator): } } +:param batch: a S

[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup

2022-07-21 Thread GitBox
potiuk commented on PR #25219: URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191670866 Thanks for the insights @SuperQ . I think might be good input for @howardyoo :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] dstandish commented on a diff in pull request #25175: Add datasets to dag dependencies view

2022-07-21 Thread GitBox
dstandish commented on code in PR #25175: URL: https://github.com/apache/airflow/pull/25175#discussion_r926861103 ## airflow/www/views.py: ## @@ -5330,6 +5330,14 @@ def _calculate_graph(self): @staticmethod def _node_dict(node_id, label, node_class): +def tru

[GitHub] [airflow] potiuk commented on issue #25177: Airflow ElasticSearch provider issue

2022-07-21 Thread GitBox
potiuk commented on issue #25177: URL: https://github.com/apache/airflow/issues/25177#issuecomment-1191667608 HA!. There you go! @PatrykKlimowicz - how about checking ot the code from #21942 and testing it it works for you :) ? -- This is an automated message from the Apache Git S

[GitHub] [airflow] eladkal commented on a diff in pull request #25164: Common SQLCheckOperators Various Functionality Update

2022-07-21 Thread GitBox
eladkal commented on code in PR #25164: URL: https://github.com/apache/airflow/pull/25164#discussion_r926859985 ## airflow/providers/common/sql/operators/sql.py: ## @@ -73,6 +79,7 @@ class SQLColumnCheckOperator(BaseSQLOperator): } } +:param batch: a

[airflow] 01/02: Fix asset compilation via setup.py (#25201)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit d8ed1975f2f5c7265a8fbaa3c0802f84f04dd61a Author: Jarek Potiuk AuthorDate: Wed Jul 20 23:06:16 2022 +0200 Fix as

[airflow] 02/02: Also compile assets in non-main (#25220)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit acc3df56c40031f04fcba733896b546a83c07fad Author: Jarek Potiuk AuthorDate: Thu Jul 21 17:58:22 2022 +0200 Also c

[airflow] branch v2-3-test updated (daf7397c34 -> acc3df56c4)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch v2-3-test in repository https://gitbox.apache.org/repos/asf/airflow.git omit daf7397c34 Fix asset compilation via setup.py (#25201) omit 51cd093b39 Also compile www assets in non-main bra

[airflow] branch main updated: Also compile assets in non-main (#25220)

2022-07-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6839813bc7 Also compile assets in non-main (#25220)

[GitHub] [airflow] potiuk merged pull request #25220: Also compile assets in non-main

2022-07-21 Thread GitBox
potiuk merged PR #25220: URL: https://github.com/apache/airflow/pull/25220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #25121: Add "Optimizing" chapter to dynamic-dags section

2022-07-21 Thread GitBox
potiuk commented on PR #25121: URL: https://github.com/apache/airflow/pull/25121#issuecomment-1191660180 > The new paragraph looks good, and it does give us something to point at to say "we told you it was experimental", I'm not sure it changes much. > > I'm not exactly sure how to ar

[GitHub] [airflow] dstandish commented on a diff in pull request #25175: Add datasets to dag dependencies view

2022-07-21 Thread GitBox
dstandish commented on code in PR #25175: URL: https://github.com/apache/airflow/pull/25175#discussion_r926843361 ## airflow/www/views.py: ## @@ -5330,6 +5330,14 @@ def _calculate_graph(self): @staticmethod def _node_dict(node_id, label, node_class): +def tru

[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup

2022-07-21 Thread GitBox
SuperQ commented on PR #25219: URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191648719 So far as I've seen, OTel is great for tracing. But everything else being tacked on is turning it into a bloated kitchen sink of a project. Trying to solve too many problems. And a lot o

[GitHub] [airflow] millin commented on issue #25177: Airflow ElasticSearch provider issue

2022-07-21 Thread GitBox
millin commented on issue #25177: URL: https://github.com/apache/airflow/issues/25177#issuecomment-1191645743 I think this mistake already [fixed here](https://github.com/apache/airflow/pull/21942/files#diff-dd898ab2ed4bca853f1ce5cf52b6fbb37d5fc3545f28967c03dc499fabd3a746R311). -- This is

[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup

2022-07-21 Thread GitBox
potiuk commented on PR #25219: URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191641848 > Eh, I would prefer to just implement Prometheus/OpenMetrics native metrics. ;-) > > I'm not really a fan of OpenTelemetry. Why ? Srsly - we have not yet implemented it but

  1   2   3   >