Re: [I] clear_task_instances support all parameters [airflow]

2023-10-27 Thread via GitHub
ephraimbuddy closed issue #35170: clear_task_instances support all parameters URL: https://github.com/apache/airflow/issues/35170 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] clear_task_instances support all parameters [airflow]

2023-10-27 Thread via GitHub
ephraimbuddy commented on issue #35170: URL: https://github.com/apache/airflow/issues/35170#issuecomment-1783726135 > > I don't think this function is intended to be used by DAG Authors and it just accidentally become a public > > cc @ephraimbuddy is that right? Yes. The functi

Re: [PR] feat(helm): add support for priorityClassName on Redis pods [airflow]

2023-10-27 Thread via GitHub
eladkal commented on code in PR #34879: URL: https://github.com/apache/airflow/pull/34879#discussion_r1375184574 ## helm_tests/other/test_redis.py: ## @@ -357,6 +357,16 @@ def test_should_use_empty_dir_on_persistence_disabled(self, redis_values, expect ) asser

Re: [I] clear_task_instances support all parameters [airflow]

2023-10-27 Thread via GitHub
eladkal commented on issue #35170: URL: https://github.com/apache/airflow/issues/35170#issuecomment-1783722915 > I don't think this function is intended to be used by DAG Authors and it just accidentally become a public cc @ephraimbuddy is that right? -- This is an automated messag

Re: [PR] Add volumeClaimTemplate for worker in Helm chart [airflow]

2023-10-27 Thread via GitHub
eladkal commented on PR #34986: URL: https://github.com/apache/airflow/pull/34986#issuecomment-1783715445 > @eladkal rebase done Tests are failing :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Airflow progressive slowness [airflow]

2023-10-27 Thread via GitHub
harshg0910 commented on issue #32928: URL: https://github.com/apache/airflow/issues/32928#issuecomment-1783709930 Hi @potiuk and @ephraimbuddy, I see "Airflow 2.7.3" milestone added to this bug, I was under the impression that it is being fixed in next bugfix release. Is it not the case (so

Re: [PR] Add volumeClaimTemplate for worker in Helm chart [airflow]

2023-10-27 Thread via GitHub
KHTee commented on PR #34986: URL: https://github.com/apache/airflow/pull/34986#issuecomment-1783701838 @eladkal rebase done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Add k8s recommended labels [airflow]

2023-10-27 Thread via GitHub
Owen-CH-Leung commented on PR #34735: URL: https://github.com/apache/airflow/pull/34735#issuecomment-1783689269 > I think you should also have to consider `Label selectors`. You mean add the newly created labels inside `label selector` ? I think `label selector` should continue to wor

Re: [PR] Add k8s recommended labels [airflow]

2023-10-27 Thread via GitHub
kimminw00 commented on PR #34735: URL: https://github.com/apache/airflow/pull/34735#issuecomment-1783677924 I think you should also have to consider Label selectors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] CLI: Variables set should allow to set description [airflow]

2023-10-27 Thread via GitHub
Zhenye-Na commented on issue #34756: URL: https://github.com/apache/airflow/issues/34756#issuecomment-1783653410 https://github.com/apache/airflow/pull/34791 is waiting for another approval, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] Improve clear_not_launched_queued_tasks call duration [airflow]

2023-10-27 Thread via GitHub
dirrao commented on PR #34985: URL: https://github.com/apache/airflow/pull/34985#issuecomment-1783640156 @uranusjr / @jedcunningham, can you review it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Refactor unneeded 'continue' jumps in models [airflow]

2023-10-27 Thread via GitHub
github-actions[bot] commented on PR #33837: URL: https://github.com/apache/airflow/pull/33837#issuecomment-1783628610 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] Update synapse.py to fix timeout bug [airflow]

2023-10-27 Thread via GitHub
github-actions[bot] closed pull request #33900: Update synapse.py to fix timeout bug URL: https://github.com/apache/airflow/pull/33900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Fix mapped task not waiting for upstream mapped to complete [airflow]

2023-10-27 Thread via GitHub
github-actions[bot] commented on PR #34138: URL: https://github.com/apache/airflow/pull/34138#issuecomment-1783628591 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [I] Contents of DAG files are assumed to have not changed [airflow]

2023-10-27 Thread via GitHub
github-actions[bot] commented on issue #34917: URL: https://github.com/apache/airflow/issues/34917#issuecomment-1783628562 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on PR #35185: URL: https://github.com/apache/airflow/pull/35185#issuecomment-1783612383 > I guess changing timeout will not work here. This looks like a real problem introduced by the change - k8s executor stops working and it should be investigated. It's rather easy

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-27 Thread via GitHub
potiuk commented on PR #35185: URL: https://github.com/apache/airflow/pull/35185#issuecomment-1783610243 I guess changing timeout will not work here. This looks like a real problem introduced by the change - k8s executor stops working and it should be investigated. It's rather easy to repr

Re: [I] Incorrect number of DAGs after update to 2.7.2 [airflow]

2023-10-27 Thread via GitHub
Taragolis closed issue #35228: Incorrect number of DAGs after update to 2.7.2 URL: https://github.com/apache/airflow/issues/35228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Incorrect number of DAGs after update to 2.7.2 [airflow]

2023-10-27 Thread via GitHub
Taragolis commented on issue #35228: URL: https://github.com/apache/airflow/issues/35228#issuecomment-1783598430 - https://github.com/apache/airflow/pull/34944 - https://github.com/apache/airflow/issues/34955 -- This is an automated message from the Apache Git Service. To respond to the

[I] Incorrect number of DAGs after update to 2.7.2 [airflow]

2023-10-27 Thread via GitHub
q121212 opened a new issue, #35228: URL: https://github.com/apache/airflow/issues/35228 ### Apache Airflow version 2.7.2 ### What happened I was update Airflow from 2.5.x to 2.7.2 in docker and on production rhel server, and in each airflows i have incorrect number of DA

(airflow) branch constraints-main updated: Updating constraints. Github run id:6671456587

2023-10-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9c2006a8be Updating con

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-27 Thread via GitHub
potiuk commented on code in PR #35218: URL: https://github.com/apache/airflow/pull/35218#discussion_r1375095744 ## setup.cfg: ## @@ -79,7 +79,7 @@ install_requires = # Update CustomTTYColoredFormatter to remove colorlog>=4.0.2, <5.0 configupdater>=3.1.1 -conne

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-27 Thread via GitHub
Kache commented on code in PR #35218: URL: https://github.com/apache/airflow/pull/35218#discussion_r1375094115 ## setup.cfg: ## @@ -79,7 +79,7 @@ install_requires = # Update CustomTTYColoredFormatter to remove colorlog>=4.0.2, <5.0 configupdater>=3.1.1 -connex

Re: [PR] remove K8S 1.24 support [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on PR #35214: URL: https://github.com/apache/airflow/pull/35214#issuecomment-1783568685 > > I think we should consider [Alibaba Cloud](https://www.alibabacloud.com/help/en/ack/product-overview/support-for-kubernetes-versions) as a major, no? Otherwise, we should repl

Re: [PR] Add OpenAI Provider [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on code in PR #35023: URL: https://github.com/apache/airflow/pull/35023#discussion_r1375065109 ## airflow/providers/openai/operators/openai.py: ## @@ -0,0 +1,73 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licens

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-27 Thread via GitHub
potiuk commented on PR #35185: URL: https://github.com/apache/airflow/pull/35185#issuecomment-1783541285 But let's see. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-27 Thread via GitHub
potiuk commented on PR #35185: URL: https://github.com/apache/airflow/pull/35185#issuecomment-1783540723 This branch is [4 commits ahead](https://github.com/hussein-awala/airflow/compare/apache:airflow:main...fix/k8s_executor_ns_override), [18 commits behind](https://github.com/hussein-awa

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-27 Thread via GitHub
potiuk commented on code in PR #35218: URL: https://github.com/apache/airflow/pull/35218#discussion_r1375066393 ## setup.cfg: ## @@ -79,7 +79,7 @@ install_requires = # Update CustomTTYColoredFormatter to remove colorlog>=4.0.2, <5.0 configupdater>=3.1.1 -conne

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-27 Thread via GitHub
Kache commented on code in PR #35218: URL: https://github.com/apache/airflow/pull/35218#discussion_r1375059245 ## setup.cfg: ## @@ -79,7 +79,7 @@ install_requires = # Update CustomTTYColoredFormatter to remove colorlog>=4.0.2, <5.0 configupdater>=3.1.1 -connex

Re: [PR] Add Cohere Provider [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1375058676 ## airflow/providers/cohere/CHANGELOG.rst: ## @@ -0,0 +1,22 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor license

[I] [helm] `strategy.type=Recreate` not fully supported by all deployments [airflow]

2023-10-27 Thread via GitHub
jiajie-chen opened a new issue, #35227: URL: https://github.com/apache/airflow/issues/35227 ### Official Helm Chart version 1.11.0 (latest released) ### Apache Airflow version v2.7.1 ### Kubernetes Version v1.27.4 (minikube v1.31.2) ### Helm Chart con

Re: [PR] Add Cohere Provider [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1375050878 ## airflow/providers/cohere/hooks/cohere.py: ## @@ -0,0 +1,82 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on PR #35185: URL: https://github.com/apache/airflow/pull/35185#issuecomment-1783518870 @potiuk, do you have an idea why the K8S executor tests failed? I tested locally with Mac arm and Linux amd, the tests finished successfully. -- This is an automated message fro

Re: [PR] Add Pinecone Provider [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on code in PR #35094: URL: https://github.com/apache/airflow/pull/35094#discussion_r1375032666 ## airflow/providers/pinecone/hooks/pinecone_hook.py: ## @@ -0,0 +1,122 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[PR] Use base aws classes in AWS Batch Operators/Sensors/Triggers [airflow]

2023-10-27 Thread via GitHub
Taragolis opened a new pull request, #35226: URL: https://github.com/apache/airflow/pull/35226 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideline

Re: [I] Auto-refresh and DAG double-trigger issues when toggling pause [airflow]

2023-10-27 Thread via GitHub
Taragolis commented on issue #35220: URL: https://github.com/apache/airflow/issues/35220#issuecomment-1783503719 You might play with different [builtin Timetables](https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/timetable.html#timetables), e.g. CronTriggerTime

Re: [I] Auto-refresh and DAG double-trigger issues when toggling pause [airflow]

2023-10-27 Thread via GitHub
Taragolis commented on issue #35220: URL: https://github.com/apache/airflow/issues/35220#issuecomment-1783498514 That is always depends on requirements, if DAG intend to use as scheduled then you need to enable it (unpause), if you need manually run newly created DAG which intends to be run

Re: [I] Auto-refresh and DAG double-trigger issues when toggling pause [airflow]

2023-10-27 Thread via GitHub
q121212 commented on issue #35220: URL: https://github.com/apache/airflow/issues/35220#issuecomment-1783455157 but this is strange, because there is no way to enable a dag that runs on a schedule so that it does not start additionally -- This is an automated message from the Apache Git Se

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-27 Thread via GitHub
hussein-awala closed pull request #35225: Fix K8S executor override config using pod_override_object URL: https://github.com/apache/airflow/pull/35225 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] corrected typos in PYPI_README.md [airflow]

2023-10-27 Thread via GitHub
potiuk closed pull request #34841: corrected typos in PYPI_README.md URL: https://github.com/apache/airflow/pull/34841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] corrected typos in PYPI_README.md [airflow]

2023-10-27 Thread via GitHub
potiuk commented on PR #34841: URL: https://github.com/apache/airflow/pull/34841#issuecomment-1783438602 I see you keep on merging the changes but you have not implemented pre-commit/the explanation from @Taragolis @KesharwaniArpita - this is a bit worrying as something of automation/bot k

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-27 Thread via GitHub
potiuk commented on code in PR #35218: URL: https://github.com/apache/airflow/pull/35218#discussion_r1374991224 ## setup.cfg: ## @@ -79,7 +79,7 @@ install_requires = # Update CustomTTYColoredFormatter to remove colorlog>=4.0.2, <5.0 configupdater>=3.1.1 -conne

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-27 Thread via GitHub
potiuk commented on code in PR #35218: URL: https://github.com/apache/airflow/pull/35218#discussion_r1374990776 ## setup.cfg: ## @@ -79,7 +79,7 @@ install_requires = # Update CustomTTYColoredFormatter to remove colorlog>=4.0.2, <5.0 configupdater>=3.1.1 -conne

Re: [I] Airflow progressive slowness [airflow]

2023-10-27 Thread via GitHub
potiuk commented on issue #32928: URL: https://github.com/apache/airflow/issues/32928#issuecomment-1783424675 No, but if someone would like to spend time on trying to analyse it it would be great. Until then - if you experience it, I recommend restarting airflow periodically. Seems like sup

Re: [PR] Add pydantic support to serde [airflow]

2023-10-27 Thread via GitHub
ribeiromarilene commented on code in PR #31565: URL: https://github.com/apache/airflow/pull/31565#discussion_r1374973990 ## airflow/serialization/serde.py: ## @@ -156,7 +156,7 @@ def serialize(o: object, depth: int = 0) -> U | None: # pydantic models are recursive if

(airflow) branch main updated: Fix subtle bug in mocking processor_agent in our tests (#35221)

2023-10-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6f3d294645 Fix subtle bug in mocking processor_agen

Re: [PR] Fix subtle bug in mocking processor_agent in our tests [airflow]

2023-10-27 Thread via GitHub
potiuk merged PR #35221: URL: https://github.com/apache/airflow/pull/35221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] Mysterious hanging of the test_retry_handling_job for sqlite on self-hosted/local env [airflow]

2023-10-27 Thread via GitHub
potiuk closed issue #35204: Mysterious hanging of the test_retry_handling_job for sqlite on self-hosted/local env URL: https://github.com/apache/airflow/issues/35204 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

(airflow) branch main updated: Add missing --no-deps when installing airflow without providers (#35224)

2023-10-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 98e7f4cc53 Add missing --no-deps when installing ai

Re: [PR] Add missing --no-deps when installing airflow without providers [airflow]

2023-10-27 Thread via GitHub
potiuk merged PR #35224: URL: https://github.com/apache/airflow/pull/35224 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-27 Thread via GitHub
hussein-awala opened a new pull request, #35225: URL: https://github.com/apache/airflow/pull/35225 Try to run the CI for #35185 from an airflow branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

(airflow) branch k8s_tests created (now 288b077b1c)

2023-10-27 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a change to branch k8s_tests in repository https://gitbox.apache.org/repos/asf/airflow.git at 288b077b1c Force multi namespace for k8s tests No new revisions were added by this update.

(airflow) branch main updated: Add operator to create jobs in Databricks (#35156)

2023-10-27 Thread pankaj
This is an automated email from the ASF dual-hosted git repository. pankaj pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new a8784e3c35 Add operator to create jobs in Databrick

Re: [I] Databricks create/reset then run-now [airflow]

2023-10-27 Thread via GitHub
pankajastro closed issue #29733: Databricks create/reset then run-now URL: https://github.com/apache/airflow/issues/29733 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Add operator to create jobs in Databricks [airflow]

2023-10-27 Thread via GitHub
pankajastro merged PR #35156: URL: https://github.com/apache/airflow/pull/35156 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

Re: [PR] Add operator to create jobs in Databricks [airflow]

2023-10-27 Thread via GitHub
tatiana commented on PR #35156: URL: https://github.com/apache/airflow/pull/35156#issuecomment-1783360492 @Lee-W @potiuk @phanikumv @pankajastro @pankajkoti This PR addresses the feedback given on #32221 and fixes the tests. The `databricks-sdk` dependency was removed since its

[PR] Add missing --no-deps when installing airflow without providers [airflow]

2023-10-27 Thread via GitHub
potiuk opened a new pull request, #35224: URL: https://github.com/apache/airflow/pull/35224 When falling back to "no-constraints" mode we missed the --no-deps to account for not-released-yet providers --- **^ Add meaningful description above** Read th

Re: [I] Auto-refresh and DAG double-trigger issues when toggling pause [airflow]

2023-10-27 Thread via GitHub
Taragolis commented on issue #35220: URL: https://github.com/apache/airflow/issues/35220#issuecomment-1783344715 Yep in this case 1 would be for scheduling execution (it could be more if catchup=True) and another for manual -- This is an automated message from the Apache Git Service. To

Re: [PR] Remove explicit annotaion for Mypy to infer type [airflow]

2023-10-27 Thread via GitHub
eladkal commented on PR #32670: URL: https://github.com/apache/airflow/pull/32670#issuecomment-1783342977 Needs another rebase after https://github.com/apache/airflow/pull/35222 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Displaying "actual" try number in TaskInstance view [airflow]

2023-10-27 Thread via GitHub
SamWheating commented on PR #34635: URL: https://github.com/apache/airflow/pull/34635#issuecomment-1783319891 Thanks for the bump, looking into this now 👀 How do you feel about the approach in general? It feels kinda hacky to me, but it technically works so 🤷 -- This is an automa

Re: [PR] remove K8S 1.24 support [airflow]

2023-10-27 Thread via GitHub
potiuk commented on PR #35214: URL: https://github.com/apache/airflow/pull/35214#issuecomment-1783309285 > I think we should consider [Alibaba Cloud](https://www.alibabacloud.com/help/en/ack/product-overview/support-for-kubernetes-versions) as a major, no? Otherwise, we should replace "unti

Re: [PR] Regenerate commands for breeze [airflow]

2023-10-27 Thread via GitHub
bolkedebruin commented on PR #35223: URL: https://github.com/apache/airflow/pull/35223#issuecomment-1783303465 Oops sorry :-). Really really though it was all green. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Regenerate commands for breeze [airflow]

2023-10-27 Thread via GitHub
potiuk commented on PR #35223: URL: https://github.com/apache/airflow/pull/35223#issuecomment-1783297340 Yep. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Regenerate commands for breeze [airflow]

2023-10-27 Thread via GitHub
potiuk closed pull request #35223: Regenerate commands for breeze URL: https://github.com/apache/airflow/pull/35223 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] Fix static checks and doc build in main [airflow]

2023-10-27 Thread via GitHub
bolkedebruin commented on PR #35222: URL: https://github.com/apache/airflow/pull/35222#issuecomment-1783297071 thanks @hussein-awala , I thought the checks read green. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Fix static checks and doc build in main [airflow]

2023-10-27 Thread via GitHub
potiuk merged PR #35222: URL: https://github.com/apache/airflow/pull/35222 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Fix static checks and doc build in main (#35222)

2023-10-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new da2fdbb760 Fix static checks and doc build in main

Re: [PR] Regenerate commands for breeze [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on PR #35223: URL: https://github.com/apache/airflow/pull/35223#issuecomment-1783295382 I opened https://github.com/apache/airflow/pull/35222 for this and for docs build error. -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] Regenerate commands for breeze [airflow]

2023-10-27 Thread via GitHub
potiuk opened a new pull request, #35223: URL: https://github.com/apache/airflow/pull/35223 The #34729 has been merged without waiting for static checks and broke the images regenerated. --- **^ Add meaningful description above** Read the **[Pull Requ

Re: [PR] Use timeout when inntializing _timeout_sec of TaskStateTrigger [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on PR #34715: URL: https://github.com/apache/airflow/pull/34715#issuecomment-1783291668 > @hussein-awala Hello, I changed to use timeout parameter and set default value of 60 if timeout has value of 'conf.getfloat("sensors", "default_timeout")'. By using timeo

Re: [PR] Fix subtle bug in mocking processor_agent in our tests [airflow]

2023-10-27 Thread via GitHub
potiuk commented on PR #35221: URL: https://github.com/apache/airflow/pull/35221#issuecomment-1783285180 This is how it manifested itself (just checked it was the same in the failing jobs in our CI): [2023-10-26T14:04:35.397+] {scheduler_job_runner.py:641} INFO - Sendi

Re: [PR] Displaying "actual" try number in TaskInstance view [airflow]

2023-10-27 Thread via GitHub
eladkal commented on PR #34635: URL: https://github.com/apache/airflow/pull/34635#issuecomment-1783281879 ping @SamWheating there is still time for this fix to be included in next release -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Fix subtle bug in mocking processor_agent in our tests [airflow]

2023-10-27 Thread via GitHub
potiuk commented on PR #35221: URL: https://github.com/apache/airflow/pull/35221#issuecomment-1783279559 > Interesting! LGTM Very much. Took me a while and intensive debugging session and comparing why it does not work for my machine but works for @aritra24 in a joint debugging sessi

[PR] Fix static checks and doc build in main [airflow]

2023-10-27 Thread via GitHub
hussein-awala opened a new pull request, #35222: URL: https://github.com/apache/airflow/pull/35222 related: #34729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] Add volumeClaimTemplate for worker in Helm chart [airflow]

2023-10-27 Thread via GitHub
eladkal commented on PR #34986: URL: https://github.com/apache/airflow/pull/34986#issuecomment-1783275371 @KHTee can you rebase and resolve conflicts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Suppress false warning when TI state is QUEUED and TI doesn't have a start_date [airflow]

2023-10-27 Thread via GitHub
eladkal commented on PR #34771: URL: https://github.com/apache/airflow/pull/34771#issuecomment-1783268991 Is this PR ready for review @kzosabe ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[PR] Fix subtle bug in mocking processor_agent in our tests [airflow]

2023-10-27 Thread via GitHub
potiuk opened a new pull request, #35221: URL: https://github.com/apache/airflow/pull/35221 Some of the scheduler tests tried to prevent DAG processor processing DAGs from "tests/dags" directory by setting processor_agent to Mock object: ```python self.job_runner.processor_agent

(airflow) branch main updated: fix(helm): set worker safeToEvict properly (#35130)

2023-10-27 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 764a0e3c95 fix(helm): set worker safeToEvict prope

Re: [PR] fix(helm): set worker safeToEvict properly [airflow]

2023-10-27 Thread via GitHub
boring-cyborg[bot] commented on PR #35130: URL: https://github.com/apache/airflow/pull/35130#issuecomment-1783266148 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [PR] fix(helm): set worker safeToEvict properly [airflow]

2023-10-27 Thread via GitHub
eladkal merged PR #35130: URL: https://github.com/apache/airflow/pull/35130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] fix: set dry_run to be optional. [airflow]

2023-10-27 Thread via GitHub
eladkal commented on code in PR #35167: URL: https://github.com/apache/airflow/pull/35167#discussion_r1374861524 ## tests/api_connexion/schemas/test_task_instance_schema.py: ## @@ -236,6 +236,22 @@ def test_success(self): } assert expected_result == result +

Re: [I] Auto-refresh and DAG double-trigger issues when toggling pause [airflow]

2023-10-27 Thread via GitHub
q121212 commented on issue #35220: URL: https://github.com/apache/airflow/issues/35220#issuecomment-1783250123 > > After push play button DAG should be triggered only one time > > It is intended behaviour if in `schedule` defined cron expression or timetable. yes, my dags have

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-27 Thread via GitHub
bolkedebruin merged PR #34729: URL: https://github.com/apache/airflow/pull/34729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

Re: [I] Auto-refresh and DAG double-trigger issues when toggling pause [airflow]

2023-10-27 Thread via GitHub
Taragolis commented on issue #35220: URL: https://github.com/apache/airflow/issues/35220#issuecomment-1783240045 > After push play button DAG should be triggered only one time It is intended behaviour if in `schedule` defined cron expression or timetable. -- This is an automated me

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1783193702 I added it to milestone 2.8.0 since the vote can be closed and already has the needed binding votes. -- This is an automated message from the Apache Git Service. To respond to t

Re: [I] Auto-refresh and DAG double-trigger issues when toggling pause [airflow]

2023-10-27 Thread via GitHub
boring-cyborg[bot] commented on issue #35220: URL: https://github.com/apache/airflow/issues/35220#issuecomment-1783186145 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] Auto-refresh and DAG double-trigger issues when toggling pause [airflow]

2023-10-27 Thread via GitHub
q121212 opened a new issue, #35220: URL: https://github.com/apache/airflow/issues/35220 ### Apache Airflow version 2.7.2 ### What happened Problems that I discovered: 1) If DAG is on pause, then when you go into it and unpause it with the button on the left, the DAG trig

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-27 Thread via GitHub
Taragolis commented on code in PR #35218: URL: https://github.com/apache/airflow/pull/35218#discussion_r1374785762 ## setup.cfg: ## @@ -79,7 +79,7 @@ install_requires = # Update CustomTTYColoredFormatter to remove colorlog>=4.0.2, <5.0 configupdater>=3.1.1 -co

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-27 Thread via GitHub
bolkedebruin commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1783139423 can I have a (final) review @potiuk @Taragolis @uranusjr ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Added to the rendering of KubernetesOperator V1VolumeMount, sub_path [airflow]

2023-10-27 Thread via GitHub
eladkal merged PR #35129: URL: https://github.com/apache/airflow/pull/35129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

(airflow) branch main updated: Added to the rendering of KubernetesOperator V1VolumeMount, sub_path (#35129)

2023-10-27 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 85f0ef35b8 Added to the rendering of KubernetesOpe

Re: [PR] Added to the rendering of KubernetesOperator V1VolumeMount, sub_path [airflow]

2023-10-27 Thread via GitHub
boring-cyborg[bot] commented on PR #35129: URL: https://github.com/apache/airflow/pull/35129#issuecomment-1783014027 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [PR] Index optimized fast path to avoid slower groupby queryplan [airflow]

2023-10-27 Thread via GitHub
joshowen commented on PR #33242: URL: https://github.com/apache/airflow/pull/33242#issuecomment-1782978113 @uranusjr @amoghrajesh @eladkal Thoughts on this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Add Snowpark Operator that allows execution of Snowflake Snowpark code [airflow]

2023-10-27 Thread via GitHub
mik-laj commented on issue #24456: URL: https://github.com/apache/airflow/issues/24456#issuecomment-1782831834 Do the problems still occur when you use the RC version of Snowflake connector? This version doesn't depends on apache-arrow, so it's less risky. -- This is an automated message

Re: [PR] remove K8S 1.24 support [airflow]

2023-10-27 Thread via GitHub
hussein-awala commented on PR #35214: URL: https://github.com/apache/airflow/pull/35214#issuecomment-1782821510 > ( USA 😅😅 ) I think we should consider [Alibaba Cloud](https://www.alibabacloud.com/help/en/ack/product-overview/support-for-kubernetes-versions) as a major, no? Otherwis

Re: [PR] remove K8S 1.24 support [airflow]

2023-10-27 Thread via GitHub
raphaelauv commented on PR #35214: URL: https://github.com/apache/airflow/pull/35214#issuecomment-1782806303 Airflow policy is "until at least 2 major ( USA 😅😅 ) cloud provider support it -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] openlineage, aws: Add OpenLineage support for AthenaOperator. [airflow]

2023-10-27 Thread via GitHub
JDarDagran commented on code in PR #35090: URL: https://github.com/apache/airflow/pull/35090#discussion_r1374477084 ## airflow/providers/amazon/aws/hooks/athena.py: ## @@ -121,7 +122,26 @@ def run_query( self.log.info("Query execution id: %s", query_execution_id)

Re: [PR] openlineage, aws: Add OpenLineage support for AthenaOperator. [airflow]

2023-10-27 Thread via GitHub
JDarDagran commented on code in PR #35090: URL: https://github.com/apache/airflow/pull/35090#discussion_r1374475519 ## airflow/providers/amazon/aws/hooks/athena.py: ## @@ -121,7 +122,26 @@ def run_query( self.log.info("Query execution id: %s", query_execution_id)

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-27 Thread via GitHub
boring-cyborg[bot] commented on PR #35218: URL: https://github.com/apache/airflow/pull/35218#issuecomment-1782799978 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [PR] Add Weaviate Provider [airflow]

2023-10-27 Thread via GitHub
utkarsharma2 commented on code in PR #35060: URL: https://github.com/apache/airflow/pull/35060#discussion_r1374470809 ## airflow/providers/weaviate/operators/weaviate.py: ## @@ -0,0 +1,87 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lic

  1   2   >