Re: [PR] Update await_pod_completion and affected unit tests [airflow]

2024-07-05 Thread via GitHub
potiuk commented on PR #40568: URL: https://github.com/apache/airflow/pull/40568#issuecomment-2211661299 > Then, we can just convert to json and get the pod status with `data["status"]["phase"]` > In my testing this results in 40x speedup, so presumably, much less CPU intensive. N

Re: [PR] enable AIRFLOW__CELERY__BROKER_URL_CMD when passwordSecretName is true [airflow]

2024-07-05 Thread via GitHub
jx2lee commented on code in PR #40270: URL: https://github.com/apache/airflow/pull/40270#discussion_r1667258370 ## chart/templates/check-values.yaml: ## @@ -30,8 +30,17 @@ The sole purpose of this yaml file is it to check the values file is consistent {{- if or (eq .Values.e

Re: [PR] enable AIRFLOW__CELERY__BROKER_URL_CMD when passwordSecretName is true [airflow]

2024-07-05 Thread via GitHub
jx2lee commented on code in PR #40270: URL: https://github.com/apache/airflow/pull/40270#discussion_r1667258360 ## chart/templates/check-values.yaml: ## @@ -30,8 +30,17 @@ The sole purpose of this yaml file is it to check the values file is consistent {{- if or (eq .Values.e

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-07-05 Thread via GitHub
trlopes1974 commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2211612922 some more info: Despite that we do have an external task id **Dependencies Blocking Task From Getting Scheduled Dependency | Reason Dagrun Running | Task instance

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-07-05 Thread via GitHub
vizeit commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2211611614 @Lee-W [this line](https://github.com/apache/airflow/blob/2eda7376f4c27df82f0cafba7699bcc46c3fcd05/airflow/providers/celery/executors/celery_executor_utils.py#L150) may require more

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-05 Thread via GitHub
dirrao commented on code in PR #39908: URL: https://github.com/apache/airflow/pull/39908#discussion_r1667236387 ## tests/_internals/forbidden_warnings.py: ## @@ -62,6 +62,11 @@ def pytest_itemcollected(self, item: pytest.Item): # Add marker at the beginning of the m

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-07-05 Thread via GitHub
Lee-W commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2211576613 > @Lee-W your suggestion of testing with AIRFLOW__CORE__EXECUTE_TASKS_NEW_PYTHON_INTERPRETER=True is not practical because the processing is 5-6 times slower. I have hundreds of dyna

Re: [PR] Dataset alias [airflow]

2024-07-05 Thread via GitHub
Lee-W commented on PR #40478: URL: https://github.com/apache/airflow/pull/40478#issuecomment-2211577179 Finally get the CI green. Will continue work on adding unit tests and docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-07-05 Thread via GitHub
vizeit commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2211552110 @Lee-W your suggestion of testing with AIRFLOW__CORE__EXECUTE_TASKS_NEW_PYTHON_INTERPRETER=True is not practical because the processing is 5-6 times slower. I have hundreds of dynam

Re: [PR] Do not dynamically determine op links for emr serverless start job operator [airflow]

2024-07-05 Thread via GitHub
dacort commented on PR #40627: URL: https://github.com/apache/airflow/pull/40627#issuecomment-2211530285 Big fan of removing the complicated code, never really liked it. The main reason I did the dynamic stuff is because the list of buttons got really long at one point, but it doesn't look

Re: [PR] Move render map index method and apply to dry run [airflow]

2024-07-05 Thread via GitHub
github-actions[bot] commented on PR #39087: URL: https://github.com/apache/airflow/pull/39087#issuecomment-2211527805 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] Add TabularVttsSensor [airflow]

2024-07-05 Thread via GitHub
github-actions[bot] closed pull request #39091: Add TabularVttsSensor URL: https://github.com/apache/airflow/pull/39091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-07-05 Thread via GitHub
vizeit commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2211493420 > I did few tests with new version 2.9.2 and have the following details with the log > > **Configuration** > > > Airflow version: 2.9.2 > > Compute: GKE > > Execu

Re: [PR] Ability to add custom facet in OpenLineage events [airflow]

2024-07-05 Thread via GitHub
anandhimurali commented on code in PR #38982: URL: https://github.com/apache/airflow/pull/38982#discussion_r1667195011 ## airflow/providers/openlineage/utils/utils.py: ## @@ -78,11 +78,38 @@ def get_job_name(task: TaskInstance) -> str: def get_custom_facets(task_instance: T

Re: [PR] Update await_pod_completion and affected unit tests [airflow]

2024-07-05 Thread via GitHub
dstandish commented on PR #40568: URL: https://github.com/apache/airflow/pull/40568#issuecomment-2211464948 > I didn't dive into the code to check if we use/need it, but between two methods that return the same result, and one of them has a cache mechanism, I prefer the one with caching.

Re: [I] Support `limit` and `offset` parameters for List task instances (batch) endpoint [airflow]

2024-07-05 Thread via GitHub
ShakaibKhan commented on issue #40038: URL: https://github.com/apache/airflow/issues/40038#issuecomment-2211463427 Looks like [page_limit and page_offset](https://github.com/apache/airflow/blob/2eda7376f4c27df82f0cafba7699bcc46c3fcd05/airflow/api_connexion/schemas/task_instance_schema.py#L10

Re: [PR] Update await_pod_completion and affected unit tests [airflow]

2024-07-05 Thread via GitHub
dstandish commented on PR #40568: URL: https://github.com/apache/airflow/pull/40568#issuecomment-2211458880 But yeah, re your comments on the first PR, I think yeah it would be a good idea to try and repro the users's situation. You would ask what their setup is, how many tasks are concurr

Re: [PR] Update await_pod_completion and affected unit tests [airflow]

2024-07-05 Thread via GitHub
dstandish commented on PR #40568: URL: https://github.com/apache/airflow/pull/40568#issuecomment-2211451593 Alright... I might have something for you @ferruzzi We can skip the V1Pod deserialization process (response -> json -> V1Pod) by adding the arg `_preload_content=False`

Re: [PR] Do not dynamically determine op links for emr serverless start job operator [airflow]

2024-07-05 Thread via GitHub
o-nikolas commented on PR #40627: URL: https://github.com/apache/airflow/pull/40627#issuecomment-2211414377 CC @dacort -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [I] EmrServerlessStartJobOperator causes dag load failure when using XComArg [airflow]

2024-07-05 Thread via GitHub
o-nikolas commented on issue #40103: URL: https://github.com/apache/airflow/issues/40103#issuecomment-2211414156 @fjmacagno @vatsrahul1001 Can you have a look at #40627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[PR] Do not dynamically determine op links for emr serverless start job operator [airflow]

2024-07-05 Thread via GitHub
o-nikolas opened a new pull request, #40627: URL: https://github.com/apache/airflow/pull/40627 The dynamic determination of which extra link to include for the EmrServerlessStartJobOperator does not work with templated fields, since that evaluation was happening at DAG parsing time. This dy

[PR] Fix class instance vs. class type in validate_database_executor_compatibility() call [airflow]

2024-07-05 Thread via GitHub
jscheffl opened a new pull request, #40626: URL: https://github.com/apache/airflow/pull/40626 While implementing AIP-69 I realized that the call to `validate_database_executor_compatibility()` is actually expecting a `type[BaseExecutor]` but in one call a class instance of `BaseExecutor` is

Re: [PR] S3 to wasb [airflow]

2024-07-05 Thread via GitHub
jroachgolf84 commented on code in PR #40511: URL: https://github.com/apache/airflow/pull/40511#discussion_r1667107509 ## tests/providers/microsoft/azure/transfers/test_s3_to_wasb.py: ## @@ -0,0 +1,333 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

Re: [PR] Microsoft Power BI operator to refresh the dataset [airflow]

2024-07-05 Thread via GitHub
ambika-garg commented on PR #40356: URL: https://github.com/apache/airflow/pull/40356#issuecomment-2211366711 Hey @dabla, I added the Triggers class to extend the operator to works in deferable mode while extending async calls to MS graph operator. now this operator could support both async

Re: [I] `DbApiHook.insert_rows` unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
eladkal commented on issue #40609: URL: https://github.com/apache/airflow/issues/40609#issuecomment-2211333518 I plan to cut provider wave tommorow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

(airflow) branch main updated: Resolve deprecations in `serialization` tests (#40625)

2024-07-05 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2eda7376f4 Resolve deprecations in `serialization`

Re: [PR] Resolve deprecations in `serialization` tests [airflow]

2024-07-05 Thread via GitHub
eladkal merged PR #40625: URL: https://github.com/apache/airflow/pull/40625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [I] `DbApiHook.insert_rows` unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
potiuk commented on issue #40609: URL: https://github.com/apache/airflow/issues/40609#issuecomment-2211201134 > For anyone who has the same problem, this should work as a temporary fix while 2.9.3 is not out. Just import these hooks instead of the ones from `airflow.providers`. You c

Re: [I] `DbApiHook.insert_rows` unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
plutaniano commented on issue #40609: URL: https://github.com/apache/airflow/issues/40609#issuecomment-2211194195 For anyone who has the same problem, this should work as a temporary fix to the problem while 2.9.3 is not out. Just import these hooks instead of the ones from `airflow.provide

[PR] Resolve deprecations in `serialization` tests [airflow]

2024-07-05 Thread via GitHub
boraberke opened a new pull request, #40625: URL: https://github.com/apache/airflow/pull/40625 related: #38642 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05

(airflow-site) branch gh-pages updated (57f6c6f378 -> baf148efdf)

2024-07-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard 57f6c6f378 Rewritten history to remove past gh-pages deployments new baf148efdf Rewritten history to

Re: [PR] Update await_pod_completion and affected unit tests [airflow]

2024-07-05 Thread via GitHub
ferruzzi commented on PR #40568: URL: https://github.com/apache/airflow/pull/40568#issuecomment-2211169419 > Unfortunately, in K8S, the pod status route returns the full pod manifest, so the result of read_namespaced_pod_status is the same as read_namespaced_pod with the same performance/co

(airflow-site) branch main updated: Update link to Python SDK (#1034)

2024-07-05 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new f6f26d3224 Update link to Python SDK (#1034)

(airflow-site) branch link deleted (was e6b73ae3cd)

2024-07-05 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch link in repository https://gitbox.apache.org/repos/asf/airflow-site.git was e6b73ae3cd Update link to Python SDK The revisions that were on this branch are still contained in other referen

Re: [PR] Update link to Python SDK [airflow-site]

2024-07-05 Thread via GitHub
eladkal merged PR #1034: URL: https://github.com/apache/airflow-site/pull/1034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airfl

[I] updated_at column in task_instance table not updating correctly [airflow]

2024-07-05 Thread via GitHub
chonein opened a new issue, #40623: URL: https://github.com/apache/airflow/issues/40623 ### Apache Airflow version 2.9.2 ### If "Other Airflow 2 version" selected, which one? Also tested on main and 2.8.0 ### What happened? When a task instance's state chang

Re: [I] updated_at column in task_instance table not updating correctly [airflow]

2024-07-05 Thread via GitHub
boring-cyborg[bot] commented on issue #40623: URL: https://github.com/apache/airflow/issues/40623#issuecomment-2211122688 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

(airflow) branch main updated: Skip checking subdagslist if taskinstance state is skipped (#40578)

2024-07-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ec5da92440 Skip checking subdagslist if taskinstanc

Re: [PR] Skip checking subdagslist if taskinstance state is skipped [airflow]

2024-07-05 Thread via GitHub
potiuk merged PR #40578: URL: https://github.com/apache/airflow/pull/40578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] Airflow API for Updating TaskInstance State to skipped is broken [airflow]

2024-07-05 Thread via GitHub
potiuk closed issue #40575: Airflow API for Updating TaskInstance State to skipped is broken URL: https://github.com/apache/airflow/issues/40575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Update await_pod_completion and affected unit tests [airflow]

2024-07-05 Thread via GitHub
potiuk commented on PR #40568: URL: https://github.com/apache/airflow/pull/40568#issuecomment-2211106549 > Unfortunately, in K8S, the pod status route returns the full pod manifest, so the result of read_namespaced_pod_status is the same as read_namespaced_pod with the same performance/cost

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-05 Thread via GitHub
potiuk commented on code in PR #39908: URL: https://github.com/apache/airflow/pull/39908#discussion_r1666946735 ## tests/_internals/forbidden_warnings.py: ## @@ -62,6 +62,11 @@ def pytest_itemcollected(self, item: pytest.Item): # Add marker at the beginning of the m

Re: [PR] Add triggered_by field to DAG Run model to distinguish the source of a trigger [airflow]

2024-07-05 Thread via GitHub
potiuk commented on PR #39165: URL: https://github.com/apache/airflow/pull/39165#issuecomment-2211091891 In the meantime - before @uranusjr responds - this one needs rebasing :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-05 Thread via GitHub
potiuk commented on code in PR #39908: URL: https://github.com/apache/airflow/pull/39908#discussion_r1666945938 ## tests/_internals/forbidden_warnings.py: ## @@ -62,6 +62,11 @@ def pytest_itemcollected(self, item: pytest.Item): # Add marker at the beginning of the m

[PR] Update link to Python SDK [airflow-site]

2024-07-05 Thread via GitHub
eladkal opened a new pull request, #1034: URL: https://github.com/apache/airflow-site/pull/1034 This is needed to avoid message `Branch master was renamed to main.` when clicking on the link -- This is an automated message from the Apache Git Service. To respond to the message, please log

(airflow-site) 01/01: Update link to Python SDK

2024-07-05 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch link in repository https://gitbox.apache.org/repos/asf/airflow-site.git commit e6b73ae3cd7eb9d1d79819bfb9abc9836ea8a814 Author: Elad Kalif <45845474+elad...@users.noreply.github.com> AuthorDate:

(airflow-site) branch link created (now e6b73ae3cd)

2024-07-05 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch link in repository https://gitbox.apache.org/repos/asf/airflow-site.git at e6b73ae3cd Update link to Python SDK This branch includes the following new commits: new e6b73ae3cd Update

(airflow-site) branch add-documentation-2024-07-02 deleted (was 0cc59e0a2d)

2024-07-05 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-07-02 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 0cc59e0a2d Add documentation for packages - 2024-07-02 The revisions that were on this b

(airflow) branch constraints-main updated: Updating constraints. Github run id:9809653314

2024-07-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 8a620981de Updating con

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-05 Thread via GitHub
dirrao commented on code in PR #39908: URL: https://github.com/apache/airflow/pull/39908#discussion_r1666863294 ## tests/_internals/forbidden_warnings.py: ## @@ -62,6 +62,11 @@ def pytest_itemcollected(self, item: pytest.Item): # Add marker at the beginning of the m

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-05 Thread via GitHub
dirrao commented on code in PR #39908: URL: https://github.com/apache/airflow/pull/39908#discussion_r1666863294 ## tests/_internals/forbidden_warnings.py: ## @@ -62,6 +62,11 @@ def pytest_itemcollected(self, item: pytest.Item): # Add marker at the beginning of the m

Re: [PR] Add timeout when watching pod events in k8s executor [airflow]

2024-07-05 Thread via GitHub
brumi1024 commented on code in PR #39551: URL: https://github.com/apache/airflow/pull/39551#discussion_r1666861403 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py: ## @@ -150,6 +148,14 @@ def _run( last_resource_version: str | None = None

(airflow) branch main updated (1dc582dba3 -> 3f0979c9f1)

2024-07-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 1dc582dba3 fix openlineage parsing dag tree with MappedOperator (#40621) add 3f0979c9f1 FIX: DbApiHook.insert_row

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
potiuk merged PR #40615: URL: https://github.com/apache/airflow/pull/40615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] task is incorrectly triggered after dynamic task mapping with BranchPythonOperator [airflow]

2024-07-05 Thread via GitHub
romsharon98 commented on issue #40620: URL: https://github.com/apache/airflow/issues/40620#issuecomment-2210953850 I see. I still don't know if it's a bug though because the reason I mention, so we will wait for third opinion. A quick action that I think of to avoid it, in branch_2_tas

Re: [PR] openlineage: fix parsing dag tree with MappedOperator [airflow]

2024-07-05 Thread via GitHub
potiuk merged PR #40621: URL: https://github.com/apache/airflow/pull/40621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: fix openlineage parsing dag tree with MappedOperator (#40621)

2024-07-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1dc582dba3 fix openlineage parsing dag tree with Ma

Re: [I] task is incorrectly triggered after dynamic task mapping with BranchPythonOperator [airflow]

2024-07-05 Thread via GitHub
raycarter commented on issue #40620: URL: https://github.com/apache/airflow/issues/40620#issuecomment-2210902237 > In the third picture the branch_2_task trigger rule set to `none_failed`,since all the previous tasks are not failed (skipped or success) it should run. the problem is a

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-05 Thread via GitHub
potiuk commented on code in PR #39908: URL: https://github.com/apache/airflow/pull/39908#discussion_r1666825880 ## tests/_internals/forbidden_warnings.py: ## @@ -62,6 +62,11 @@ def pytest_itemcollected(self, item: pytest.Item): # Add marker at the beginning of the m

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
potiuk commented on PR #40615: URL: https://github.com/apache/airflow/pull/40615#issuecomment-2210893494 AH. ... one small static check to fix! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(airflow-site) branch gh-pages updated (b3452f98d9 -> 57f6c6f378)

2024-07-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard b3452f98d9 Rewritten history to remove past gh-pages deployments new 57f6c6f378 Rewritten history to

[PR] openlineage: fix parsing dag tree with MappedOperator [airflow]

2024-07-05 Thread via GitHub
kacpermuda opened a new pull request, #40621: URL: https://github.com/apache/airflow/pull/40621 Recently added AirflowJobFacet contains taskTree that can't handle MappedOperators. This PR fixes it and adds tests with MappedOperator and TaskFlow operators to make sure we can hand

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
potiuk commented on code in PR #40615: URL: https://github.com/apache/airflow/pull/40615#discussion_r1666792851 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -181,19 +182,21 @@ def __init__(self, *args, schema: str | None = None, log_sql: bool = True, **kwa "

Re: [PR] DockerOperator TaskFlow - improve logs [airflow]

2024-07-05 Thread via GitHub
antoinetavant commented on PR #39620: URL: https://github.com/apache/airflow/pull/39620#issuecomment-2210839082 I've updated the PR by adding a regression test and only addressing the first point, as the second has been addressed. -- This is an automated message from the Apache Git Servic

Re: [PR] Updating airflow download links to CDN based links for all past releases [airflow-site]

2024-07-05 Thread via GitHub
potiuk commented on PR #1033: URL: https://github.com/apache/airflow-site/pull/1033#issuecomment-2210826306 Fantastic ! Thanks ! Let's see when docs build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Updating airflow download links to CDN based links for all past releases [airflow-site]

2024-07-05 Thread via GitHub
potiuk merged PR #1033: URL: https://github.com/apache/airflow-site/pull/1033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Skip checking subdagslist if taskinstance state is skipped [airflow]

2024-07-05 Thread via GitHub
romsharon98 commented on PR #40578: URL: https://github.com/apache/airflow/pull/40578#issuecomment-2210815520 static check fail. better work with [pre-commit](https://github.com/apache/airflow/blob/main/contributing-docs/03_contributors_quick_start.rst#configuring-pre-commit), it will fi

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
dabla commented on code in PR #40615: URL: https://github.com/apache/airflow/pull/40615#discussion_r147928 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -181,19 +182,21 @@ def __init__(self, *args, schema: str | None = None, log_sql: bool = True, **kwa "r

Re: [I] task is incorrectly triggered after dynamic task mapping with BranchPythonOperator [airflow]

2024-07-05 Thread via GitHub
romsharon98 commented on issue #40620: URL: https://github.com/apache/airflow/issues/40620#issuecomment-2210798782 In the third picture the branch_2_task trigger rule set to `none_failed`,since all the previous tasks are not failed (skipped or success) it should run. -- This is an automa

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
dabla commented on PR #40615: URL: https://github.com/apache/airflow/pull/40615#issuecomment-2210780200 > Some tests to fix, but it looks good :) Some compatibility tests are failing but I don't understand why -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] enable AIRFLOW__CELERY__BROKER_URL_CMD when passwordSecretName is true [airflow]

2024-07-05 Thread via GitHub
romsharon98 commented on code in PR #40270: URL: https://github.com/apache/airflow/pull/40270#discussion_r1666738938 ## helm_tests/other/test_redis.py: ## @@ -143,7 +143,9 @@ def test_redis_by_chart_password(self, executor): self.assert_broker_url_env(k8s_obj_by_key)

Re: [PR] Updating airflow download links to CDN based links for all past releases [airflow-site]

2024-07-05 Thread via GitHub
amoghrajesh commented on PR #1033: URL: https://github.com/apache/airflow-site/pull/1033#issuecomment-2210748373 cc @potiuk this is a huge one (my ide almost gave up), but it should be correct -- This is an automated message from the Apache Git Service. To respond to the message, please l

[PR] Updating airflow download links to CDN based links for all past releases [airflow-site]

2024-07-05 Thread via GitHub
amoghrajesh opened a new pull request, #1033: URL: https://github.com/apache/airflow-site/pull/1033 related: https://github.com/apache/airflow/pull/40618 Yeah this one is slightly massive! -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Add dark theme, by @prefers-color-theme [airflow-site]

2024-07-05 Thread via GitHub
mayo7e commented on issue #803: URL: https://github.com/apache/airflow-site/issues/803#issuecomment-2210745067 @porwalshreyaa Alright, I'll take some time to go over the documentation and share my thoughts on how we can divide the tasks. -- This is an automated message from the Apache Git

Re: [I] Add dark theme, by @prefers-color-theme [airflow-site]

2024-07-05 Thread via GitHub
mayo7e commented on issue #803: URL: https://github.com/apache/airflow-site/issues/803#issuecomment-2210734373 @porwalshreyaa Perhaps you can point me to the MDN documentation in question so I can get up to speed. I'm not sure what you mean by "divide pages." According to @potiuk , all tha

[I] task is incorrectly triggered after dynamic task mapping with BranchPythonOperator [airflow]

2024-07-05 Thread via GitHub
raycarter opened a new issue, #40620: URL: https://github.com/apache/airflow/issues/40620 ### Apache Airflow version 2.9.2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? We need to use BranchPythonOperator and dynamic

Re: [I] Add dark theme, by @prefers-color-theme [airflow-site]

2024-07-05 Thread via GitHub
porwalshreyaa commented on issue #803: URL: https://github.com/apache/airflow-site/issues/803#issuecomment-2210724188 @mayo7e I haven't created a branch yet, I just read the mdn doc. Maybe we can do divide pages you can do one page completely by yourself and I'll do another, or you have som

Re: [I] `DbApiHook.insert_rows` unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
plutaniano commented on issue #40609: URL: https://github.com/apache/airflow/issues/40609#issuecomment-2210720790 Thanks a lot, guys. Really appreciate the attention given to this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] Add dark theme, by @prefers-color-theme [airflow-site]

2024-07-05 Thread via GitHub
mayo7e commented on issue #803: URL: https://github.com/apache/airflow-site/issues/803#issuecomment-2210718154 @porwalshreyaa happy to work with you on this! Is there a feature branch you already created that I can clone? If you have the tasks defined, I can get started with a task right

Re: [I] Add dark theme, by @prefers-color-theme [airflow-site]

2024-07-05 Thread via GitHub
porwalshreyaa commented on issue #803: URL: https://github.com/apache/airflow-site/issues/803#issuecomment-2210715407 @potiuk Thanks! I've told my friends that I am working on this, if I don't get to merge, I'll be in trouble 😂 -- This is an automated message from the Apache Git Service.

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-05 Thread via GitHub
dirrao commented on code in PR #39908: URL: https://github.com/apache/airflow/pull/39908#discussion_r196299 ## tests/_internals/forbidden_warnings.py: ## @@ -62,6 +62,11 @@ def pytest_itemcollected(self, item: pytest.Item): # Add marker at the beginning of the m

Re: [I] Can not cognize type of model's real type in mypy [airflow]

2024-07-05 Thread via GitHub
potiuk closed issue #40617: Can not cognize type of model's real type in mypy URL: https://github.com/apache/airflow/issues/40617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Add dark theme, by @prefers-color-theme [airflow-site]

2024-07-05 Thread via GitHub
potiuk commented on issue #803: URL: https://github.com/apache/airflow-site/issues/803#issuecomment-2210703886 Ups. FAT FINGERS -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [I] Add dark theme, by @prefers-color-theme [airflow-site]

2024-07-05 Thread via GitHub
porwalshreyaa commented on issue #803: URL: https://github.com/apache/airflow-site/issues/803#issuecomment-2210686811 @potiuk why did you unassign me? I am still working on it and he asked to join me, maybe you could just assign us both? -- This is an automated message from the Apache Git

Re: [I] `DbApiHook.insert_rows` unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
potiuk commented on issue #40609: URL: https://github.com/apache/airflow/issues/40609#issuecomment-2210672431 cc: @bjankie1 :D ^^ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [I] `DbApiHook.insert_rows` unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
potiuk commented on issue #40609: URL: https://github.com/apache/airflow/issues/40609#issuecomment-2210670713 > Completely agree on that, it will cost as well in performance as in money (disk). Hopefully, in the future, [AIP-59](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-59+Per

Re: [PR] Updating airflow download links to CDN based links [airflow]

2024-07-05 Thread via GitHub
potiuk merged PR #40618: URL: https://github.com/apache/airflow/pull/40618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Updating airflow download links to CDN based links (#40618)

2024-07-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 77a9850556 Updating airflow download links to CDN b

Re: [I] `DbApiHook.insert_rows` unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
dabla commented on issue #40609: URL: https://github.com/apache/airflow/issues/40609#issuecomment-2210660972 > Ah also logging migtht be expensive (money) as well :D depends on whether you use remote logging solution and whether it charges "per message". Completely agree on that, it w

Re: [I] Add dark theme, by @prefers-color-theme [airflow-site]

2024-07-05 Thread via GitHub
potiuk commented on issue #803: URL: https://github.com/apache/airflow-site/issues/803#issuecomment-2210657014 BTW. The PR for dark mode got the title of "PR of the month" in June unanimously, so you are not the only want that considered that as a blocker - see https://lists.apache.org/thr

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
dabla commented on code in PR #40615: URL: https://github.com/apache/airflow/pull/40615#discussion_r147928 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -181,19 +182,21 @@ def __init__(self, *args, schema: str | None = None, log_sql: bool = True, **kwa "r

[PR] Updating airflow download links to CDN based links [airflow]

2024-07-05 Thread via GitHub
amoghrajesh opened a new pull request, #40618: URL: https://github.com/apache/airflow/pull/40618 Updating download from sources links to https://dlcdn.apache.org/ from the earlier links. Airbyte for example: https://github.com/apache/airflow/assets/35884252/e8fd497e

[I] Can't not cognize type of model's real type in mypy [airflow]

2024-07-05 Thread via GitHub
barking-code opened a new issue, #40617: URL: https://github.com/apache/airflow/issues/40617 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.0 ### What happened? After updati

Re: [I] Can't not cognize type of model's real type in mypy [airflow]

2024-07-05 Thread via GitHub
boring-cyborg[bot] commented on issue #40617: URL: https://github.com/apache/airflow/issues/40617#issuecomment-2210635894 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] gitSync: add extraEnvFrom to git-sync containers to support proxy settings from k8s secret [airflow]

2024-07-05 Thread via GitHub
rasulkarimov commented on code in PR #39031: URL: https://github.com/apache/airflow/pull/39031#discussion_r142790 ## chart/templates/_helpers.yaml: ## @@ -198,6 +198,7 @@ If release name contains chart name it will be used as a full name. image: {{ template "git_sync_ima

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
dabla commented on code in PR #40615: URL: https://github.com/apache/airflow/pull/40615#discussion_r139771 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -181,19 +182,21 @@ def __init__(self, *args, schema: str | None = None, log_sql: bool = True, **kwa "r

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
dabla commented on code in PR #40615: URL: https://github.com/apache/airflow/pull/40615#discussion_r139243 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -181,19 +182,21 @@ def __init__(self, *args, schema: str | None = None, log_sql: bool = True, **kwa "r

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-05 Thread via GitHub
dirrao commented on PR #39908: URL: https://github.com/apache/airflow/pull/39908#issuecomment-2210614224 I tried adding the ignore warning in `pyproject.toml` file. However it's not working. `'ignore:Timer and timing metrics publish in seconds were deprecated. It is enabled by default f

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
potiuk commented on code in PR #40615: URL: https://github.com/apache/airflow/pull/40615#discussion_r113418 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -181,19 +182,21 @@ def __init__(self, *args, schema: str | None = None, log_sql: bool = True, **kwa "

Re: [PR] FIX: DbApiHook.insert_rows unnecessarily restarting connections [airflow]

2024-07-05 Thread via GitHub
potiuk commented on code in PR #40615: URL: https://github.com/apache/airflow/pull/40615#discussion_r112349 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -181,19 +182,21 @@ def __init__(self, *args, schema: str | None = None, log_sql: bool = True, **kwa "

  1   2   >