Re: [PR] Python3.12 [airflow]

2024-01-24 Thread via GitHub
Taragolis commented on PR #36755: URL: https://github.com/apache/airflow/pull/36755#issuecomment-1907614960 > Fixed I think errors with caled_once in a few tests. We also have to exclude cassandra for now: Hmm.. I looks like 3.29 version of [cassandra-driver](https://github.com/datas

Re: [PR] Add Qdrant Provider [airflow]

2024-01-24 Thread via GitHub
eladkal commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1464499920 ## docs/apache-airflow-providers-qdrant/connections.rst: ## @@ -0,0 +1,58 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
romsharon98 commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1907625589 Validate changes in: https://github.com/apache/airflow/pull/36911 https://github.com/apache/airflow/pull/36905 https://github.com/apache/airflow/pull/36663 https://gi

Re: [PR] k8s spark operator compatibility [airflow]

2024-01-24 Thread via GitHub
ruslanguns commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1907626900 This PR is very important for us, we are start working with Airflow and Spark and this additional means a lot. I really encourage the reviewers to review this. Thanks in advanced:

Re: [PR] Implement deferrable mode for CreateHyperparameterTuningJobOperator [airflow]

2024-01-24 Thread via GitHub
moiseenkov commented on code in PR #36594: URL: https://github.com/apache/airflow/pull/36594#discussion_r1464505265 ## airflow/providers/google/cloud/hooks/vertex_ai/hyperparameter_tuning_job.py: ## @@ -413,3 +428,106 @@ def delete_hyperparameter_tuning_job( metadat

Re: [PR] Implement deferrable mode for CreateHyperparameterTuningJobOperator [airflow]

2024-01-24 Thread via GitHub
moiseenkov commented on code in PR #36594: URL: https://github.com/apache/airflow/pull/36594#discussion_r1464505785 ## airflow/providers/google/cloud/hooks/vertex_ai/hyperparameter_tuning_job.py: ## @@ -413,3 +428,106 @@ def delete_hyperparameter_tuning_job( metadat

Re: [PR] Add Qdrant Provider [airflow]

2024-01-24 Thread via GitHub
Anush008 commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1464507774 ## docs/apache-airflow-providers-qdrant/connections.rst: ## @@ -0,0 +1,58 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
VladaZakharova commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1907633506 Hi! #36473 works as well, thank you :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
moiseenkov commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1907634824 #36276 works as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Add Qdrant Provider [airflow]

2024-01-24 Thread via GitHub
Anush008 commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1464513380 ## docs/apache-airflow-providers-qdrant/connections.rst: ## @@ -0,0 +1,58 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
AchimGaedkeLynker commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1907641660 https://github.com/apache/airflow/pull/36828 works as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] heartbeat recovery message [airflow]

2024-01-24 Thread via GitHub
Bowrna commented on code in PR #34457: URL: https://github.com/apache/airflow/pull/34457#discussion_r1464523088 ## airflow/jobs/job.py: ## @@ -126,7 +127,19 @@ def executor(self): def heartrate(self) -> float: return Job._heartrate(self.job_type) -def is_aliv

Re: [PR] Add Qdrant Provider [airflow]

2024-01-24 Thread via GitHub
Anush008 commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1464513380 ## docs/apache-airflow-providers-qdrant/connections.rst: ## @@ -0,0 +1,58 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

Re: [PR] Add Qdrant Provider [airflow]

2024-01-24 Thread via GitHub
Anush008 commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1464513380 ## docs/apache-airflow-providers-qdrant/connections.rst: ## @@ -0,0 +1,58 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

Re: [PR] Add Qdrant Provider [airflow]

2024-01-24 Thread via GitHub
Anush008 commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1464513380 ## docs/apache-airflow-providers-qdrant/connections.rst: ## @@ -0,0 +1,58 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

[I] Missing properties on logger [airflow]

2024-01-24 Thread via GitHub
Grub4K opened a new issue, #36992: URL: https://github.com/apache/airflow/issues/36992 ### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Using `'b' in sys.stdout.mode` (see h

Re: [I] Missing properties on logger [airflow]

2024-01-24 Thread via GitHub
boring-cyborg[bot] commented on issue #36992: URL: https://github.com/apache/airflow/issues/36992#issuecomment-1907666343 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Python3.12 [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36755: URL: https://github.com/apache/airflow/pull/36755#issuecomment-1907672936 > Hmm.. I looks like 3.29 version of [cassandra-driver](https://github.com/datastax/python-driver/blob/master/CHANGELOG.rst) should support Python 3.12 In a way. Looking to the is

Re: [PR] Fix successful Apache Druid task submissions reported as failed [airflow]

2024-01-24 Thread via GitHub
listik commented on PR #36813: URL: https://github.com/apache/airflow/pull/36813#issuecomment-1907698338 > Thanks for adding the test! Thank you for the feedback and patience! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
avkirilishin commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1464581551 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Add ability to automatically set DAG to off after X times it failed sequentially [airflow]

2024-01-24 Thread via GitHub
BasPH commented on code in PR #36935: URL: https://github.com/apache/airflow/pull/36935#discussion_r1464581185 ## airflow/models/dagrun.py: ## @@ -498,6 +498,37 @@ def fetch_task_instances( tis = tis.where(TI.task_id.in_(task_ids)) return session.scalars(ti

(airflow) branch python3.12 updated (d6c8e89370 -> 9198f00bbf)

2024-01-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch python3.12 in repository https://gitbox.apache.org/repos/asf/airflow.git omit d6c8e89370 python 3.12 support add 9198f00bbf python 3.12 support This update added new revisions after und

[PR] Remove duplicated definitions of random volumes [airflow]

2024-01-24 Thread via GitHub
potiuk opened a new pull request, #36993: URL: https://github.com/apache/airflow/pull/36993 Docker Compose 2.24 started to error out on duplicated (but otherwise identical) volume definitions coming from multiple docker compose files. We had urandom volume defined not only in base yaml but

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
Lee-W commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1907775136 Tested https://github.com/apache/airflow/pull/36586, https://github.com/apache/airflow/pull/36578, https://github.com/apache/airflow/pull/36550, https://github.com/apache/airflow/pu

[I] Button on UI to pause all DAGs [airflow]

2024-01-24 Thread via GitHub
yshkp opened a new issue, #36994: URL: https://github.com/apache/airflow/issues/36994 ### Description Slider button on UI of airflow to pause/unpause all DAGs. ### Use case/motivation During deployment, every time there is a use case to pause all dags to not run tasks in

Re: [I] Button on UI to pause all DAGs [airflow]

2024-01-24 Thread via GitHub
boring-cyborg[bot] commented on issue #36994: URL: https://github.com/apache/airflow/issues/36994#issuecomment-1907778200 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36969: URL: https://github.com/apache/airflow/pull/36969#issuecomment-1907851710 > @potiuk I started looking at it too, will complete by EOD, but I am not very much in favour of labelling the docs as -title.rst, when only title.rst would be enough, any specific reaso

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1907880496 I am not sure we want it. If you reallly want to do worker-only deployment, you should create your own helm/resources -that will be way simpler and our chart is not meant to be as versat

Re: [PR] k8s spark operator compatibility [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1907882516 Can you rebase it please @blcksrx ? There are some conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Fix spelling of `airflow` [airflow]

2024-01-24 Thread via GitHub
potiuk merged PR #36682: URL: https://github.com/apache/airflow/pull/36682 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Fix spelling of `airflow` (#36682)

2024-01-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 396402d1f9 Fix spelling of `airflow` (#36682) 39640

Re: [PR] Fixing status icon alignment for various views [airflow]

2024-01-24 Thread via GitHub
amoghrajesh commented on PR #36804: URL: https://github.com/apache/airflow/pull/36804#issuecomment-1907929883 @bbovenzi can you take a look at this when you have some time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[PR] Create DataprocStartClusterOperator and DataprocStopClusterOperator [airflow]

2024-01-24 Thread via GitHub
molcay opened a new pull request, #36996: URL: https://github.com/apache/airflow/pull/36996 Implement DataprocStartClusterOperator and DataprocStopClusterOperator Also, Add the following logic to DataprocCreateClusterOperator: - if the cluster already exists, and it is in stopped state,

Re: [PR] Create DataprocStartClusterOperator and DataprocStopClusterOperator [airflow]

2024-01-24 Thread via GitHub
boring-cyborg[bot] commented on PR #36996: URL: https://github.com/apache/airflow/pull/36996#issuecomment-1908022721 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] add description to queue_when [airflow]

2024-01-24 Thread via GitHub
devscheffer opened a new pull request, #36997: URL: https://github.com/apache/airflow/pull/36997 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideli

Re: [PR] add description to queue_when [airflow]

2024-01-24 Thread via GitHub
boring-cyborg[bot] commented on PR #36997: URL: https://github.com/apache/airflow/pull/36997#issuecomment-1908097600 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
renzepost commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1908146019 #36817 works as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-24 Thread via GitHub
aritra24 commented on code in PR #36969: URL: https://github.com/apache/airflow/pull/36969#discussion_r1464930407 ## README.md: ## @@ -426,13 +427,23 @@ might decide to add additional limits (and justify them with comment). ## Contributing -Want to help build Apache Airflo

Re: [PR] Standardize deprecations in providers [part1] [airflow]

2024-01-24 Thread via GitHub
kacpermuda commented on PR #36876: URL: https://github.com/apache/airflow/pull/36876#issuecomment-1908197262 Hey, is there anything else that i can do to make this PR merged? Pinging those who at least saw it 😄 @eladkal @potiuk @o-nikolas -- This is an automated message from the Apache

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465001706 ## airflow/providers/teradata/transfers/teradata_to_teradata.py: ## @@ -0,0 +1,91 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more c

[I] Kubernetes Executor Task Leak [airflow]

2024-01-24 Thread via GitHub
smhood opened a new issue, #36998: URL: https://github.com/apache/airflow/issues/36998 ### Apache Airflow version 2.8.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Scheduler stops processing DAGs and moving them to

Re: [I] Kubernetes Executor Task Leak [airflow]

2024-01-24 Thread via GitHub
boring-cyborg[bot] commented on issue #36998: URL: https://github.com/apache/airflow/issues/36998#issuecomment-1908253016 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465010521 ## airflow/providers/teradata/example_dags/example_teradata_to_teradata_transfer_operator.py: ## @@ -0,0 +1,158 @@ +# Review Comment: system tests are defined

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465010521 ## airflow/providers/teradata/example_dags/example_teradata_to_teradata_transfer_operator.py: ## @@ -0,0 +1,158 @@ +# Review Comment: system tests are defined

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465012016 ## airflow/providers/teradata/example_dags/example_teradata_operator.py: ## @@ -0,0 +1,132 @@ +# Review Comment: system tests are defined under [system ](htt

Re: [PR] Remove duplicated definitions of random volumes [airflow]

2024-01-24 Thread via GitHub
hussein-awala merged PR #36993: URL: https://github.com/apache/airflow/pull/36993 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated: Remove duplicated definitions of random volumes (#36993)

2024-01-24 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 390eacb01a Remove duplicated definitions of r

(airflow) branch python3.12 updated (9198f00bbf -> 84763a9a09)

2024-01-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch python3.12 in repository https://gitbox.apache.org/repos/asf/airflow.git discard 9198f00bbf python 3.12 support add 0e17f40884 Update faq.rst with (hopefully) clearer description of start_

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
eladkal commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465092920 ## airflow/providers/teradata/example_dags/example_teradata_operator.py: ## @@ -0,0 +1,132 @@ +# Review Comment: All example dags will be converted to system test

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
jedcunningham commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1908397607 @potiuk I've actually come around to this idea over time. You can sorta do it today with scaling those components to 0, but you end up with all that stuff in k8s anyways. Plus it

[PR] Enable '_enable_tcp_keepalive' functionality for GKEPodHook [airflow]

2024-01-24 Thread via GitHub
MaksYermak opened a new pull request, #36999: URL: https://github.com/apache/airflow/pull/36999 In this PR I have enabled TCP keepalive functionality for GKEPodHook. We have the same functionality for KubernetesHook. Also this PR is a fix for this issue https://github.com/ap

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1908412000 > @potiuk I've actually come around to this idea over time. You can sorta do it today with scaling those components to 0, but you end up with all that stuff in k8s anyways. Plus it also

(airflow-site) branch main updated: Better link to "enable_task_context_logger" in 2.8 blog (#941)

2024-01-24 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 431634d6b5 Better link to "enable_task_

(airflow-site) branch better_logging_link deleted (was d0d07769a9)

2024-01-24 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch better_logging_link in repository https://gitbox.apache.org/repos/asf/airflow-site.git was d0d07769a9 Better link to "enable_task_context_logger" in 2.8 blog The revisions that were

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
jedcunningham commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1908415574 We also need test coverage for this 👍. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Better link to "enable_task_context_logger" in 2.8 blog [airflow-site]

2024-01-24 Thread via GitHub
jedcunningham merged PR #941: URL: https://github.com/apache/airflow-site/pull/941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@a

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
hussein-awala commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1908431833 Tested my changes, they look good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Invalid kube-config file. Expected key current-context in kube-config when using deferrable=True [airflow]

2024-01-24 Thread via GitHub
hussein-awala commented on issue #34644: URL: https://github.com/apache/airflow/issues/34644#issuecomment-1908440168 > This PR was a bug fix, also there was another reason for it (will explain more later). For the second reason, here is it: https://www.cve.org/CVERecord?id=CVE-2023-5

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465178743 ## airflow/providers/teradata/transfers/teradata_to_teradata.py: ## @@ -0,0 +1,91 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more c

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465179800 ## airflow/providers/teradata/transfers/teradata_to_teradata.py: ## @@ -0,0 +1,91 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more c

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465186558 ## airflow/providers/teradata/example_dags/example_teradata_operator.py: ## @@ -0,0 +1,132 @@ +# Review Comment: Got it. Do you want to remove example dags no

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1465186558 ## airflow/providers/teradata/example_dags/example_teradata_operator.py: ## @@ -0,0 +1,132 @@ +# Review Comment: Got it. Is it required to remove these now or

(airflow-site) branch gh-pages updated (7e37a0485d -> 179ae0bee4)

2024-01-24 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard 7e37a0485d Rewritten history to remove past gh-pages deployments new 179ae0bee4 Rewritten history to

Re: [I] Tasks taking too long time after 2.7.0 Airflow update [airflow]

2024-01-24 Thread via GitHub
BenoCharlo commented on issue #33688: URL: https://github.com/apache/airflow/issues/33688#issuecomment-1908481958 @pankajkoti I've tried the 2.6.3 version today. A lot of dags are starting to run as expected. It has gotten a bit better but I am still experiencing the same delay problem for

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-24 Thread via GitHub
SatishChGit commented on PR #36953: URL: https://github.com/apache/airflow/pull/36953#issuecomment-1908478949 > Partial review for now Addressed the review comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

(airflow) branch refactor-contributing-docs updated (db304088a5 -> 5e0a500f9e)

2024-01-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit db304088a5 Split contributing docs to multiple files add 1c14767638 Fix google operators han

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36969: URL: https://github.com/apache/airflow/pull/36969#discussion_r1465206732 ## README.md: ## @@ -426,13 +427,23 @@ might decide to add additional limits (and justify them with comment). ## Contributing -Want to help build Apache Airflow?

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36969: URL: https://github.com/apache/airflow/pull/36969#discussion_r1465208033 ## README.md: ## @@ -426,13 +427,23 @@ might decide to add additional limits (and justify them with comment). ## Contributing -Want to help build Apache Airflow?

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36969: URL: https://github.com/apache/airflow/pull/36969#issuecomment-1908494200 Applied the comments from @aritra24 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Implement deferrable mode for CreateHyperparameterTuningJobOperator [airflow]

2024-01-24 Thread via GitHub
moiseenkov commented on PR #36594: URL: https://github.com/apache/airflow/pull/36594#issuecomment-1908515483 @Lee-W , hi, Could you please review my fixes corresponding to your comments? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36969: URL: https://github.com/apache/airflow/pull/36969#issuecomment-1908553450 Also - just to add @amoghrajesh - nice way of showing what I explained about sequence: https://github.com/apache/airflow/assets/595491/b6bfae42-8fba-4efc-b671-93a48ce9c259";>

Re: [PR] Fix `log_query` to format SQL statement correctly in `AthenaOperator` [airflow]

2024-01-24 Thread via GitHub
ferruzzi commented on PR #36962: URL: https://github.com/apache/airflow/pull/36962#issuecomment-1908616937 @dirrao I didn't add any test cases because it's just reformatting a string. Could you suggest a test case you'd like to see? I thought of two: one with a single-line query and one

[PR] Adding Amazon Neptune Hook and Operators [airflow]

2024-01-24 Thread via GitHub
ellisms opened a new pull request, #37000: URL: https://github.com/apache/airflow/pull/37000 --- closes #28289 * Added Neptune hook * Added NeptuneStartDbClusterOperator and NeptuneStopDbClusterOperator * Added custom waiter for checking the status of a cluster, rather than

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
vizeit commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1908637320 #36922 tested. It is working for configmaps mounted as volume but not for configmaps mounted as environment variable. I was troubleshooting the issue and it appears that I will need

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2024-01-24 Thread via GitHub
o-nikolas commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1465379421 ## tests/providers/amazon/aws/operators/test_emr_serverless.py: ## Review Comment: tests were added -- This is an automated message from the Apache Git Ser

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2024-01-24 Thread via GitHub
o-nikolas commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1465381768 ## airflow/providers/amazon/aws/operators/emr.py: ## @@ -1352,6 +1391,101 @@ def on_kill(self) -> None: check_interval_seconds=self.waiter_delay,

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2024-01-24 Thread via GitHub
o-nikolas commented on PR #34225: URL: https://github.com/apache/airflow/pull/34225#issuecomment-1908698764 Only one conversation left open that needs resolution and then I think we're ready to merge! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Enable '_enable_tcp_keepalive' functionality for GKEPodHook [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36999: URL: https://github.com/apache/airflow/pull/36999#issuecomment-1908718898 How about a unit test? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
kajanajustin commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1908729539 @potiuk @jedcunningham Let me know what you guys think about the doc edits I made. Let me know if I need to add any additional descriptions -- This is an automated message from t

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1908762897 Minimal, but good enough for me :) . Still a few tests would be needed to approve that one. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Add Qdrant Provider [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1465429667 ## docs/apache-airflow-providers-qdrant/connections.rst: ## @@ -0,0 +1,58 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor l

Re: [PR] Add Qdrant Provider [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1465430866 ## docs/apache-airflow-providers-qdrant/connections.rst: ## @@ -0,0 +1,58 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor l

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
kajanajustin commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1908795831 @potiuk anything else needed from me at this time? I assume you guys have the testing handled from here -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1908828287 > @potiuk anything else needed from me at this time? I assume you guys have the testing handled from here No - unit tests are always part of the PRs. We have separately run tests f

Re: [PR] Added ability to enable/disable scheduler and webserver [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36991: URL: https://github.com/apache/airflow/pull/36991#issuecomment-1908830072 also you have static checks failing and you should fix them. For that you can take a look at this split doc in my PR https://github.com/apache/airflow/blob/refactor-contributing-docs/con

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465501481 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465501481 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465501481 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465501481 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465501481 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465501481 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465505953 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465501481 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

Re: [PR] Handle SystemExit raised in the task. [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36986: URL: https://github.com/apache/airflow/pull/36986#discussion_r1465501481 ## airflow/models/taskinstance.py: ## @@ -2393,6 +2393,12 @@ def _run_raw_task( self.handle_failure(e, test_mode, context, session=session)

[PR] 36888-Fix k8 configmap issue in 7.14.0rc1 [airflow]

2024-01-24 Thread via GitHub
vizeit opened a new pull request, #37001: URL: https://github.com/apache/airflow/pull/37001 This change is to fix issue from [Provider cncf.kubernetes: 7.14.0rc1](https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/7.14.0rc1) testing for the PR #36922. After troubleshooting th

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
vizeit commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1908937247 > #36922 tested. It is working for configmaps mounted as volume but not for configmaps mounted as environment variable. I was troubleshooting the issue and it appears that I will ne

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
potiuk commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1908944188 > Opened https://github.com/apache/airflow/pull/37001 with the fix and unit tested Cool. Yep. that's ok. In Airlfow `PR` is the `unit of work` - not issue, issue is mostly ad

Re: [PR] Add celery pid suffix to fix duplicate pid issue [airflow]

2024-01-24 Thread via GitHub
potiuk commented on code in PR #36794: URL: https://github.com/apache/airflow/pull/36794#discussion_r1465565377 ## airflow/cli/commands/celery_command.py: ## @@ -170,7 +170,7 @@ def worker(args): "--loglevel", celery_log_level, "--pidfile", -wo

Re: [PR] Migrate provider-specific executor docs to providers [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #34809: URL: https://github.com/apache/airflow/pull/34809#issuecomment-1908962757 Just to comment on the errors @RNHTTR and guide you a bit.. I think all the _api errors are coming from those first few errors here. Usually they appear when the main package fail

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-24 Thread via GitHub
flolas commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1908966335 Tested https://github.com/apache/airflow/issues/36948#:~:text=Implement%20AthenaSQLHook%20(%2336171). Work as expected. Env: Amazon MWAA [v2.7.2](https://pypi.python.org/py

  1   2   >