(airflow) branch main updated: Fix import future annotations in venv jinja template (#40208)

2024-06-13 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d5a75446a6 Fix import future annotations in venv

Re: [PR] Fix import future annotations in venv jinja template [airflow]

2024-06-13 Thread via GitHub
potiuk merged PR #40208: URL: https://github.com/apache/airflow/pull/40208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix import future annotations in venv jinja template [airflow]

2024-06-13 Thread via GitHub
potiuk commented on PR #40208: URL: https://github.com/apache/airflow/pull/40208#issuecomment-2167217269 > It passes all the tests, but maybe there's a problem with the wrong label. @potiuk Could you change it and run action again? Ah yeah - now it's not a "docker provider" test - it

Re: [PR] Add slightly more detailed guidance about upgrading to the docs [airflow]

2024-06-13 Thread via GitHub
potiuk commented on code in PR #40227: URL: https://github.com/apache/airflow/pull/40227#discussion_r1639258479 ## docs/apache-airflow/installation/upgrading.rst: ## @@ -57,6 +57,10 @@ when you choose to upgrade airflow via their UI. How to upgrade == +Reinstall

Re: [PR] Update `Dag.test()` to run with an executor if desired [airflow]

2024-06-13 Thread via GitHub
potiuk commented on code in PR #40205: URL: https://github.com/apache/airflow/pull/40205#discussion_r1639254923 ## airflow/models/dag.py: ## @@ -2952,6 +2952,21 @@ def add_logger_if_needed(ti: TaskInstance): # Instead of starting a scheduler, we run the minimal

Re: [PR] Update `Dag.test()` to run with an executor if desired [airflow]

2024-06-13 Thread via GitHub
potiuk commented on code in PR #40205: URL: https://github.com/apache/airflow/pull/40205#discussion_r1639253344 ## contributing-docs/testing/unit_tests.rst: ## @@ -1440,6 +1440,29 @@ to **ignore**, e.g. set ``PYTHONWARNINGS`` environment variable to ``ignore``. pytest

Re: [PR] AIP-61 - Add executor field to the task instance API [airflow]

2024-06-13 Thread via GitHub
uranusjr commented on PR #40034: URL: https://github.com/apache/airflow/pull/40034#issuecomment-2167196651 Please rebase first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Chart: set `workers.safeToEvict` default to False [airflow]

2024-06-13 Thread via GitHub
jedcunningham opened a new pull request, #40229: URL: https://github.com/apache/airflow/pull/40229 This is a safer default for our workers. This can be safe to set to true if you have a long enough `workers.terminationGracePeriodSeconds` set, but what is "long enough" is very

Re: [PR] AIP-61 - Add executor field to the task instance API [airflow]

2024-06-13 Thread via GitHub
dstandish commented on code in PR #40034: URL: https://github.com/apache/airflow/pull/40034#discussion_r1639128433 ## airflow/api_connexion/openapi/v1.yaml: ## @@ -5499,7 +5517,14 @@ components: items: type: string description: The value can be

Re: [PR] Fix import future annotations in venv jinja template [airflow]

2024-06-13 Thread via GitHub
phi-friday commented on PR #40208: URL: https://github.com/apache/airflow/pull/40208#issuecomment-2167083374 It passes all the tests, but maybe there's a problem with the wrong label. @potiuk Could you change it and run action again? -- This is an automated message from the Apache Git

[PR] Fix typo in tasks.rst [airflow]

2024-06-13 Thread via GitHub
bangjiehan opened a new pull request, #40228: URL: https://github.com/apache/airflow/pull/40228 --- **^ Add meaningful description above** Read the **[Pull Request

Re: [I] cannot use `get_current_context()` in a `@task.virtualenv` task [airflow]

2024-06-13 Thread via GitHub
gGonz commented on issue #34158: URL: https://github.com/apache/airflow/issues/34158#issuecomment-2167066979 > @gGonz > > > Can we at least have some workaround or example, the docs on the links pointed above doesn't provide any hint of how to fix this issue. > > Even though

[PR] Add slightly more detailed guidance about upgrading to the docs [airflow]

2024-06-13 Thread via GitHub
merobi-hub opened a new pull request, #40227: URL: https://github.com/apache/airflow/pull/40227 The docs lack specific guidance on upgrading Airflow. Although the path might be obvious to advanced users, more details would be useful to many -- if the views

Re: [PR] AIP-61 - Add executor field to the task instance API [airflow]

2024-06-13 Thread via GitHub
syedahsn commented on code in PR #40034: URL: https://github.com/apache/airflow/pull/40034#discussion_r1639097200 ## airflow/www/static/js/dag/details/taskInstance/Details.tsx: ## @@ -53,6 +53,7 @@ const Details = ({ gridInstance, taskInstance, group }: Props) => { const

Re: [PR] AIP-61 - Add executor field to the task instance API [airflow]

2024-06-13 Thread via GitHub
syedahsn commented on PR #40034: URL: https://github.com/apache/airflow/pull/40034#issuecomment-2167037850 It looks like the failing test was resolved in #39899. This should be good to merge @o-nikolas? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Add more detailed guidance on upgrading Airflow [airflow-site]

2024-06-13 Thread via GitHub
merobi-hub closed pull request #1024: Add more detailed guidance on upgrading Airflow URL: https://github.com/apache/airflow-site/pull/1024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Fix import future annotations in docker decorator [airflow]

2024-06-13 Thread via GitHub
phi-friday commented on PR #40208: URL: https://github.com/apache/airflow/pull/40208#issuecomment-2167011779 Changed the location, as it affects venv operator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Fix import future annotations in docker decorator [airflow]

2024-06-13 Thread via GitHub
phi-friday commented on PR #40208: URL: https://github.com/apache/airflow/pull/40208#issuecomment-2167005287 First, I changed the test to only work with `airflow>=2.10`. But looking at the output, I don't think it's a change to `airflow[docker]`. Should I change the test location

Re: [PR] Redacting the sensitive env variables in env_vars for KPO [airflow]

2024-06-13 Thread via GitHub
github-actions[bot] commented on PR #39003: URL: https://github.com/apache/airflow/pull/39003#issuecomment-2166987584 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [I] Airflow auto-refresh on UI [airflow]

2024-06-13 Thread via GitHub
github-actions[bot] commented on issue #39209: URL: https://github.com/apache/airflow/issues/39209#issuecomment-2166987555 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [I] smtp connection doc should mention non-credentials field will not be used for notification [airflow]

2024-06-13 Thread via GitHub
boring-cyborg[bot] commented on issue #40226: URL: https://github.com/apache/airflow/issues/40226#issuecomment-2166982367 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] smtp connection doc should mention non-credentials field will not be used for notification [airflow]

2024-06-13 Thread via GitHub
chulkilee opened a new issue, #40226: URL: https://github.com/apache/airflow/issues/40226 ### What do you see as an issue? [Email Configuration](https://airflow.apache.org/docs/apache-airflow/stable/howto/email-config.html) mentions > To configure SMTP settings, checkout the

[PR] Add more detailed guidance on upgrading Airflow [airflow-site]

2024-06-13 Thread via GitHub
merobi-hub opened a new pull request, #1024: URL: https://github.com/apache/airflow-site/pull/1024 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] cannot use `get_current_context()` in a `@task.virtualenv` task [airflow]

2024-06-13 Thread via GitHub
pedro-cf commented on issue #34158: URL: https://github.com/apache/airflow/issues/34158#issuecomment-2166899120 @gGonz > Can we at least have some workaround or example, the docs on the links pointed above doesn't provide any hint of how to fix this issue. Even though it is not

Re: [I] cannot use `get_current_context()` in a `@task.virtualenv` task [airflow]

2024-06-13 Thread via GitHub
mziwisky commented on issue #34158: URL: https://github.com/apache/airflow/issues/34158#issuecomment-2166889638 The original post on this issue offers a workaround -- access context properties by making them parameters of your task function. e.g. ```python

[I] clear dag_run with filter on status bug [airflow]

2024-06-13 Thread via GitHub
raphaelauv opened a new issue, #40225: URL: https://github.com/apache/airflow/issues/40225 ### Apache Airflow version 2.9.2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? when you set a filter on status of dags like

Re: [PR] PoC of AIP-69 - A first task can be called remote [airflow]

2024-06-13 Thread via GitHub
jscheffl commented on code in PR #40224: URL: https://github.com/apache/airflow/pull/40224#discussion_r1638977511 ## docs/apache-airflow-providers-remote/celery_executor.rst: ## Review Comment: Note: Don't take this RST serious - it is at the moment a copy from celery as

Re: [PR] PoC of AIP-69 - A first task can be called remote [airflow]

2024-06-13 Thread via GitHub
jscheffl commented on code in PR #40224: URL: https://github.com/apache/airflow/pull/40224#discussion_r1638976841 ## airflow/serialization/serialized_objects.py: ## Review Comment: Note: This extension is needed to allow new Pydantic class mappings from Remote Provider

Re: [PR] PoC of AIP-69 - A first task can be called remote [airflow]

2024-06-13 Thread via GitHub
jscheffl commented on code in PR #40224: URL: https://github.com/apache/airflow/pull/40224#discussion_r1638976054 ## airflow/providers/remote/provider.yaml: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] PoC of AIP-69 - A first task can be called remote [airflow]

2024-06-13 Thread via GitHub
jscheffl commented on code in PR #40224: URL: https://github.com/apache/airflow/pull/40224#discussion_r1638975094 ## airflow/providers/remote/_start_remote_worker.sh: ## Review Comment: This is a temporary script for development testing to start the remote worker.

Re: [PR] PoC of AIP-69 - A first task can be called remote [airflow]

2024-06-13 Thread via GitHub
jscheffl commented on code in PR #40224: URL: https://github.com/apache/airflow/pull/40224#discussion_r1638974529 ## airflow/providers/remote/__init__.py: ## @@ -0,0 +1,39 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] PoC of AIP-69 - A first task can be called remote [airflow]

2024-06-13 Thread via GitHub
jscheffl commented on code in PR #40224: URL: https://github.com/apache/airflow/pull/40224#discussion_r1638973865 ## airflow/example_dags/example_subdag_operator.py: ## Review Comment: Note: I (temporarily) deleted this example as it makes compile issues with internal API

Re: [PR] PoC of AIP-69 - A first task can be called remote [airflow]

2024-06-13 Thread via GitHub
jscheffl commented on code in PR #40224: URL: https://github.com/apache/airflow/pull/40224#discussion_r1638973326 ## airflow/api_internal/internal_api_call.py: ## Review Comment: Note: This change is needed allowing to re-route the Internal API calls to another endpoint

[PR] PoC of AIP-69 - A first task can be called remote [airflow]

2024-06-13 Thread via GitHub
jscheffl opened a new pull request, #40224: URL: https://github.com/apache/airflow/pull/40224 This PR is a DRAFT to further discuss technical details on AIP-69 direction. DO NOT MERGE - but please feedback on it. Please treat this as a PoC, it is only partly working. It can be used

Re: [I] cannot use `get_current_context()` in a `@task.virtualenv` task [airflow]

2024-06-13 Thread via GitHub
gGonz commented on issue #34158: URL: https://github.com/apache/airflow/issues/34158#issuecomment-2166805892 > Contrary to popular believes, asking whether there are any updates on issue that does not have any update and not providing any more information and contributing findings, do no t

Re: [PR] Fix import future annotations in docker decorator [airflow]

2024-06-13 Thread via GitHub
potiuk commented on PR #40208: URL: https://github.com/apache/airflow/pull/40208#issuecomment-2166755163 The test will have to be skipped on airflow pre 2.10 because it will not work there (See failing compatibility tests) and ways how to deal with the compatibility issues:

Re: [PR] openlineage: migrate OpenLineage provider to V2 facets. [airflow]

2024-06-13 Thread via GitHub
JDarDagran commented on PR #39530: URL: https://github.com/apache/airflow/pull/39530#issuecomment-2166627546 yep, waiting for better times :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Adding a new model to Airflow lib [airflow-client-python]

2024-06-13 Thread via GitHub
potiuk closed issue #113: Adding a new model to Airflow lib URL: https://github.com/apache/airflow-client-python/issues/113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Adding a new model to Airflow lib [airflow-client-python]

2024-06-13 Thread via GitHub
potiuk commented on issue #113: URL: https://github.com/apache/airflow-client-python/issues/113#issuecomment-2166690191 Generally speaking airflow is not designed for the API to be extendable. There are no recipes to do it and it's not somethig that is even remotely guaranteed to work in

Re: [PR] Implement ChainXComArg [airflow]

2024-06-13 Thread via GitHub
potiuk commented on PR #40172: URL: https://github.com/apache/airflow/pull/40172#issuecomment-2166670508 Maybe just `xcom_chain()` or `chain_xcom()` ? That would remove confusion and keep itertools relation - at the expense of somewhat redundancy. -- This is an automated message from the

Re: [PR] Databricks: stop including user names in `list_jobs` [airflow]

2024-06-13 Thread via GitHub
potiuk commented on PR #40178: URL: https://github.com/apache/airflow/pull/40178#issuecomment-216515 Almost. BTW. Installing pre-commit and running them on the change should auto-fix all static errors. -- This is an automated message from the Apache Git Service. To respond to the

(airflow) branch main updated (2272ea246a -> 67798b2b8a)

2024-06-13 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 2272ea246a AIP-64: Add UI endpoint for task instance history (#40221) add 67798b2b8a Fix Scheduler

Re: [PR] Fix Scheduler restarting due to too many completed pods in cluster [airflow]

2024-06-13 Thread via GitHub
ephraimbuddy merged PR #40183: URL: https://github.com/apache/airflow/pull/40183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] AIP-64: Add UI endpoint for task instance history [airflow]

2024-06-13 Thread via GitHub
ephraimbuddy merged PR #40221: URL: https://github.com/apache/airflow/pull/40221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: AIP-64: Add UI endpoint for task instance history (#40221)

2024-06-13 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2272ea246a AIP-64: Add UI endpoint for

Re: [I] Google Cloud BigQuery Transfer Operator to Google Cloud Storage misleading documentation [airflow]

2024-06-13 Thread via GitHub
potiuk commented on issue #40217: URL: https://github.com/apache/airflow/issues/40217#issuecomment-2166643306 Feel free to fix it. Though you do not need to open an issue to fix it with PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Fixed issue of new dag getting old dataset events. [airflow]

2024-06-13 Thread via GitHub
RNHTTR commented on PR #39603: URL: https://github.com/apache/airflow/pull/39603#issuecomment-2166639298 > Looks good to me. Pending @RNHTTR’s comments. LGTM as well, but I'm unable to mark the comments as resolved :\ -- This is an automated message from the Apache Git Service. To

Re: [I] Run example_bash_decorator dag failed [airflow]

2024-06-13 Thread via GitHub
potiuk closed issue #40216: Run example_bash_decorator dag failed URL: https://github.com/apache/airflow/issues/40216 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[I] Adding a new model to Airflow lib [airflow-client-python]

2024-06-13 Thread via GitHub
kjsbot opened a new issue, #113: URL: https://github.com/apache/airflow-client-python/issues/113 Hi I would like to add a new model to Airflow. I'm building a wrapper class around Airflow API client along with some new endpoints and API methods so I would like to create a new model. Is

Re: [PR] Update `Dag.test()` to run with an executor if desired [airflow]

2024-06-13 Thread via GitHub
vincbeck commented on PR #40205: URL: https://github.com/apache/airflow/pull/40205#issuecomment-2166470921 > Looks very cool. Apart of getting support for other commands for `keep-env-variables` I think it we also need a small chapter in `unit_tests.rst` describing this kind of test

[PR] Add error handling for encoding the dag runs [airflow]

2024-06-13 Thread via GitHub
molcay opened a new pull request, #40222: URL: https://github.com/apache/airflow/pull/40222 This PR is related to adding an error handling for the `circular reference` on `/object/grid_data` endpoint which is used while rendering the DAG detail page. Previously it was like the

Re: [PR] Update `Dag.test()` to run with an executor if desired [airflow]

2024-06-13 Thread via GitHub
vincbeck commented on code in PR #40205: URL: https://github.com/apache/airflow/pull/40205#discussion_r1638332941 ## airflow/models/dag.py: ## @@ -2952,6 +2952,21 @@ def add_logger_if_needed(ti: TaskInstance): # Instead of starting a scheduler, we run the minimal

Re: [PR] Fix Scheduler restarting due to too many completed pods in cluster [airflow]

2024-06-13 Thread via GitHub
jedcunningham commented on code in PR #40183: URL: https://github.com/apache/airflow/pull/40183#discussion_r1638413499 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py: ## @@ -271,14 +273,9 @@ def process_status( elif status == "Succeeded":

Re: [PR] openlineage: migrate OpenLineage provider to V2 facets. [airflow]

2024-06-13 Thread via GitHub
potiuk commented on PR #39530: URL: https://github.com/apache/airflow/pull/39530#issuecomment-2165802148 My proposal is to wait for LAZY CONSENSUS on compat provider and implement it (See https://lists.apache.org/thread/hjmrv1wc2rxhkvozpvccs4mhgwg5sw2f) -- This is an automated message

Re: [I] Changing role name in access_control doesn't remove permission from previous role [airflow]

2024-06-13 Thread via GitHub
zachliu commented on issue #17508: URL: https://github.com/apache/airflow/issues/17508#issuecomment-2165784872 > > i'm on 2.9.2 and experiencing this too > > whenever the creation of a dag with `dag.access_control = {"Role_A": {"can_edit", "can_read"}}` precedes the creation of `Role_A`,

Re: [PR] Update `Dag.test()` to run with an executor if desired [airflow]

2024-06-13 Thread via GitHub
vincbeck commented on code in PR #40205: URL: https://github.com/apache/airflow/pull/40205#discussion_r1638372046 ## dev/breeze/src/airflow_breeze/commands/testing_commands_config.py: ## @@ -96,6 +96,7 @@ "--mount-sources",

(airflow) branch main updated: doc: metrics allow_list complet example (#40120)

2024-06-13 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 205ad57d04 doc: metrics allow_list complet example

Re: [PR] doc: metrics allow_list complet example [airflow]

2024-06-13 Thread via GitHub
potiuk merged PR #40120: URL: https://github.com/apache/airflow/pull/40120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] AIP-64: Add UI endpoint for task instance history [airflow]

2024-06-13 Thread via GitHub
ephraimbuddy opened a new pull request, #40221: URL: https://github.com/apache/airflow/pull/40221 This adds UI endpoint for task instance history -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-06-13 Thread via GitHub
potiuk commented on PR #39908: URL: https://github.com/apache/airflow/pull/39908#issuecomment-2165717567 > I don't have the access to wiki. I will remind you on this. If you want access, send me your email in Slack -- This is an automated message from the Apache Git Service. To

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-06-13 Thread via GitHub
dirrao commented on PR #39908: URL: https://github.com/apache/airflow/pull/39908#issuecomment-2165652486 > Also if we do that - I think that should be added to our Airflow 3 discussions as a topic to discuss -

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-06-13 Thread via GitHub
dirrao commented on PR #39908: URL: https://github.com/apache/airflow/pull/39908#issuecomment-2165606051 > I **think** if we want to implement this consistency, it should be Airlfow 3 feature to have it on by default and possibly we should raise a future deprecation warning informing that

(airflow) branch openlineage-process-execution updated (55e579286a -> 8cbb8bcf73)

2024-06-13 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a change to branch openlineage-process-execution in repository https://gitbox.apache.org/repos/asf/airflow.git discard 55e579286a openlineage: execute extraction and message sending in separate process

Re: [PR] Refactor Datapipeline operators and fix system tests [airflow]

2024-06-13 Thread via GitHub
VladaZakharova commented on PR #39716: URL: https://github.com/apache/airflow/pull/39716#issuecomment-2165136150 Hi @eladkal ! Can you please check changes again here? :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Data type convertion for extra in connection [airflow]

2024-06-13 Thread via GitHub
boring-cyborg[bot] commented on issue #40220: URL: https://github.com/apache/airflow/issues/40220#issuecomment-2165478889 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Data type convertion for extra in connection [airflow]

2024-06-13 Thread via GitHub
ThePsyjo opened a new issue, #40220: URL: https://github.com/apache/airflow/issues/40220 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.7.3 ### What happened? Web-interface >

Re: [PR] Fix dag task scheduled and queued duration metrics [airflow]

2024-06-13 Thread via GitHub
htpawel commented on PR #37936: URL: https://github.com/apache/airflow/pull/37936#issuecomment-2165475953 @eladkal seems like once again there is some flaky test failing, not related to my change. Could someone override its result or rerun only it or fix? Otherwise we will never merge such

[PR] fix bigquery_to_gcs documentation [airflow]

2024-06-13 Thread via GitHub
joonvena opened a new pull request, #40219: URL: https://github.com/apache/airflow/pull/40219 Currently the documentation states Importing files and also suggest that the file would be imported from GCS to BigQuery instead of what the operator actually does which is exporting single

Re: [I] Google Cloud BigQuery Transfer Operator to Google Cloud Storage misleading documentation [airflow]

2024-06-13 Thread via GitHub
boring-cyborg[bot] commented on issue #40217: URL: https://github.com/apache/airflow/issues/40217#issuecomment-2165451102 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Google Cloud BigQuery Transfer Operator to Google Cloud Storage misleading documentation [airflow]

2024-06-13 Thread via GitHub
joonvena opened a new issue, #40217: URL: https://github.com/apache/airflow/issues/40217 ### What do you see as an issue? https://airflow.apache.org/docs/apache-airflow-providers-google/stable/operators/transfer/bigquery_to_gcs.html Currently this documentation gives

Re: [I] Run example_bash_decorator dag failed [airflow]

2024-06-13 Thread via GitHub
boring-cyborg[bot] commented on issue #40216: URL: https://github.com/apache/airflow/issues/40216#issuecomment-2165440191 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Run example_bash_decorator dag failed [airflow]

2024-06-13 Thread via GitHub
yangyulely opened a new issue, #40216: URL: https://github.com/apache/airflow/issues/40216 ### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? get_file_stats error occurs when

(airflow) branch main updated: Resolve deprecations in `LatestOnlyOperator` tests (#40181)

2024-06-13 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new feb8307472 Resolve deprecations in

Re: [PR] Resolve deprecations in `LatestOnlyOperator` tests [airflow]

2024-06-13 Thread via GitHub
eladkal merged PR #40181: URL: https://github.com/apache/airflow/pull/40181 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Tasks are in queued state for a longer time and executor slots are exhausted often [airflow]

2024-06-13 Thread via GitHub
ephraimbuddy commented on issue #38968: URL: https://github.com/apache/airflow/issues/38968#issuecomment-2165175179 Can anyone try this patch https://github.com/apache/airflow/pull/40183 for the scheduler restarting issue? -- This is an automated message from the Apache Git Service. To

Re: [PR] Refactor Datapipeline operators and fix system tests [airflow]

2024-06-13 Thread via GitHub
eladkal commented on code in PR #39716: URL: https://github.com/apache/airflow/pull/39716#discussion_r1638055028 ## airflow/contrib/operators/__init__.py: ## @@ -174,6 +174,14 @@ "airflow.providers.google.cloud.operators.dataflow.DataflowTemplatedJobStartOperator"

Re: [PR] Fix dag task scheduled and queued duration metrics [airflow]

2024-06-13 Thread via GitHub
htpawel commented on PR #37936: URL: https://github.com/apache/airflow/pull/37936#issuecomment-2165354889 @ferruzzi who has write access and can merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Implement ChainXComArg [airflow]

2024-06-13 Thread via GitHub
uranusjr commented on PR #40172: URL: https://github.com/apache/airflow/pull/40172#issuecomment-2165328765 I don’t like `append` personally since in Python it’s a method that modifies the object in-place, not creating a new object. (Same for `extend` although nobody mentions it yet.) I

Re: [PR] Databricks: stop including user names in `list_jobs` [airflow]

2024-06-13 Thread via GitHub
stephenpurcell-db commented on PR #40178: URL: https://github.com/apache/airflow/pull/40178#issuecomment-2165322451 @potiuk thanks! I think the tests should pass now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Make task go to terminal state from triggerer without needing a worker [airflow]

2024-06-13 Thread via GitHub
pankajastro commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1637940056 ## airflow/triggers/base.py: ## @@ -135,3 +148,105 @@ def __eq__(self, other): if isinstance(other, TriggerEvent): return other.payload ==

[I] List Dag Run broken link to dag_run [airflow]

2024-06-13 Thread via GitHub
raphaelauv opened a new issue, #40214: URL: https://github.com/apache/airflow/issues/40214 ### Apache Airflow version 2.9.2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? if I click on a dag_run on the list dag run

Re: [PR] Make task go to terminal state from triggerer without needing a worker [airflow]

2024-06-13 Thread via GitHub
Lee-W commented on code in PR #40084: URL: https://github.com/apache/airflow/pull/40084#discussion_r1637896854 ## airflow/exceptions.py: ## @@ -372,14 +372,20 @@ class TaskDeferred(BaseException): Signal an operator moving to deferred state. Special exception raised

Re: [I] Integrate Snowflake Notebook Scheduling with Apache Airflow [airflow]

2024-06-13 Thread via GitHub
Lee-W commented on issue #40167: URL: https://github.com/apache/airflow/issues/40167#issuecomment-2165144309 Sounds good to me  @jacobcbeaudin I'll close it for now. But please feel free to reopen it -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] adjust the task execution order [airflow]

2024-06-13 Thread via GitHub
eladkal commented on issue #40213: URL: https://github.com/apache/airflow/issues/40213#issuecomment-2165144496 > I acknowledge that I am a maintainer/committer of the Apache Airflow project. You are not a mantainer of the project. Please don't use this option. If you have a bug or

Re: [I] adjust the task execution order [airflow]

2024-06-13 Thread via GitHub
eladkal closed issue #40213: adjust the task execution order URL: https://github.com/apache/airflow/issues/40213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Integrate Snowflake Notebook Scheduling with Apache Airflow [airflow]

2024-06-13 Thread via GitHub
Lee-W closed issue #40167: Integrate Snowflake Notebook Scheduling with Apache Airflow URL: https://github.com/apache/airflow/issues/40167 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Integrate Snowflake Notebook Scheduling with Apache Airflow [airflow]

2024-06-13 Thread via GitHub
blcksrx commented on issue #40167: URL: https://github.com/apache/airflow/issues/40167#issuecomment-2165110238 @jacobcbeaudin @Lee-W I understand, let's close this issue then, once Snowflake announced the official documentation then we'll continue -- This is an automated message from

Re: [I] Update the Search Ads 360 hook to Call the New Reporting API [airflow]

2024-06-13 Thread via GitHub
Taishan314 commented on issue #39942: URL: https://github.com/apache/airflow/issues/39942#issuecomment-2165104401 @molcay Thank you very much! Do you know if the version will be released before the 30th June? -- This is an automated message from the Apache Git Service. To respond to the

(airflow) branch main updated (2149b4dbee -> fa65a20d4a)

2024-06-13 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 2149b4dbee Ensures DAG params order regardless of backend (#40156) add fa65a20d4a local task job: add

Re: [PR] local task job: add overtime mechanism, to not kill task prematurely when auxiliary processes are running [airflow]

2024-06-13 Thread via GitHub
mobuchowski merged PR #39890: URL: https://github.com/apache/airflow/pull/39890 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Are views and authorized views handled by the `BigQueryUpdateTableOperator` [airflow]

2024-06-13 Thread via GitHub
boring-cyborg[bot] commented on issue #40212: URL: https://github.com/apache/airflow/issues/40212#issuecomment-2164964090 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] adjust the task execution order [airflow]

2024-06-13 Thread via GitHub
tongxin880120 opened a new issue, #40213: URL: https://github.com/apache/airflow/issues/40213 ### Body ![image](https://github.com/apache/airflow/assets/37608505/00556b36-7e1a-4247-b74f-d75df71299f4) Hi, I would like to ask a problem. I want to adjust one task first to

Re: [PR] Fix import future annotations in docker decorator [airflow]

2024-06-13 Thread via GitHub
phi-friday commented on code in PR #40208: URL: https://github.com/apache/airflow/pull/40208#discussion_r1637835611 ## airflow/utils/python_virtualenv_script.jinja2: ## @@ -16,7 +16,9 @@ specific language governing permissions and limitations under the License. -#} - +{%

[I] Are views and authorized views handled by the `BigQueryUpdateTableOperator` [airflow]

2024-06-13 Thread via GitHub
MokhFn opened a new issue, #40212: URL: https://github.com/apache/airflow/issues/40212 ### What do you see as an issue? This [documentation](https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-google/operators/cloud/bigquery.rst) clarifies that views can be

Re: [PR] Adding Glue Data Quality Rule Recommendation Run [airflow]

2024-06-13 Thread via GitHub
gopidesupavan commented on PR #40014: URL: https://github.com/apache/airflow/pull/40014#issuecomment-2164907796 Thanks for confirming. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Fix Scheduler restarting due to too many completed pods in cluster [airflow]

2024-06-13 Thread via GitHub
ephraimbuddy commented on code in PR #40183: URL: https://github.com/apache/airflow/pull/40183#discussion_r1637720232 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py: ## @@ -269,21 +271,6 @@ def process_status( (pod_name, namespace,

Re: [PR] Fix Scheduler restarting due to too many completed pods in cluster [airflow]

2024-06-13 Thread via GitHub
ephraimbuddy commented on code in PR #40183: URL: https://github.com/apache/airflow/pull/40183#discussion_r1637718476 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py: ## @@ -139,7 +139,9 @@ def _run( ) -> str | None:

Re: [I] Provide access privilege required for `cncf.kubernetes` operators/executors [airflow]

2024-06-13 Thread via GitHub
potiuk commented on issue #40210: URL: https://github.com/apache/airflow/issues/40210#issuecomment-2164755137 Sure. Marked it as good-first-issue and you are free to contribute it, otherwise it will have to wait for someone to volunteer, pick it up and contribute. -- This is an

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-06-13 Thread via GitHub
potiuk commented on PR #39908: URL: https://github.com/apache/airflow/pull/39908#issuecomment-2164664097 WHY: This is generally a `serious deployment` change for a number of users - but since it's a deployment and not `DAG authoring` change, it's likely a good candidate for one of the

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-06-13 Thread via GitHub
potiuk commented on PR #39908: URL: https://github.com/apache/airflow/pull/39908#issuecomment-2164647130 I **think** if we want to implement this consistency, it should be Airlfow 3 feature to have it on by default and possibly we should raise a future deprecation warning informing that it

  1   2   >