dirrao opened a new pull request, #36882:
URL: https://github.com/apache/airflow/pull/36882
What happened
When the K8 executor is unable to launch the worker pod due to permissions
issues or an invalid namespace. The K8 executor keep trying to launch the
worker pod and the errors rem
shohamy7 commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1459077084
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -436,7 +436,7 @@ def sync(self) -> None:
except ApiException as e:
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1459273598
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -436,7 +436,7 @@ def sync(self) -> None:
except ApiException as e:
eladkal commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1901185211
Just to clarify this also solves
https://github.com/apache/airflow/issues/35792 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
jedcunningham commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1901186517
> Just to clarify this also solves #35792 ?
Yes, it would.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
jedcunningham commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1901190184
It might be worth adding a note in the changelog about this behavior change,
so folks can reevaluate if they need to enable/increase retries.
--
This is an automated message fro
eladkal commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1901194151
> It might be worth adding a note in the changelog about this behavior
change, so folks can reevaluate if they need to enable/increase retries.
Agree. @dirrao can you please add n
hussein-awala commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1460108973
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
dirrao commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1901747231
> > It might be worth adding a note in the changelog about this behavior
change, so folks can reevaluate if they need to enable/increase retries.
>
> Agree. @dirrao can you please
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1460214073
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1460214073
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1460214073
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
shohamy7 commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1460536555
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
sel
hussein-awala commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1460539740
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1460550276
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
jedcunningham commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1462584433
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1462668330
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
hussein-awala commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1463107125
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1463426729
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
shohamy7 commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1463452920
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
sel
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1463613047
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1463613047
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
amoghrajesh commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1464287734
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1465782241
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
jedcunningham commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1466816732
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
jedcunningham commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1466816732
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
hussein-awala commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1468587468
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
hterik commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1469164669
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1469165496
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
chenyair commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1914986603
I see you added retires counter. What do you think about custom delay
between each retry of exceeded quota also? My issue is the high rate of
requests to Kubernetes API and currently i
dirrao commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1915929151
> I see you added retires counter. What do you think about custom delay
between each retry of exceeded quota also? My issue is the high rate of
requests to Kubernetes API and currently i
jedcunningham commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1470543337
##
airflow/providers/cncf/kubernetes/provider.yaml:
##
@@ -350,6 +350,15 @@ config:
type: string
example: ~
default: ""
+ task_p
jedcunningham commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1916006948
> > I see you added retires counter. What do you think about custom delay
between each retry of exceeded quota also? My issue is the high rate of
requests to Kubernetes API and cu
dirrao commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1916298950
> > > I see you added retires counter. What do you think about custom delay
between each retry of exceeded quota also? My issue is the high rate of
requests to Kubernetes API and current
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1470762084
##
airflow/providers/cncf/kubernetes/provider.yaml:
##
@@ -350,6 +350,15 @@ config:
type: string
example: ~
default: ""
+ task_publish_
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1470762084
##
airflow/providers/cncf/kubernetes/provider.yaml:
##
@@ -350,6 +350,15 @@ config:
type: string
example: ~
default: ""
+ task_publish_
devscheffer commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1921445378
I had similar problems and thought about something like that
```
from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import
KubernetesPodOperator
from airflo
dirrao commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1922753144
> I had similar problems and thought about something like that
Ok. I would suggest to use dedicated pool slots per namespace. pool slots
should depicts the namespace resources. So,
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1476162147
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
hussein-awala commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1476561460
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,19 +438,35 @@ def sync(self) -> None:
)
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1476927531
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,19 +438,35 @@ def sync(self) -> None:
)
sel
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1476927807
##
airflow/providers/cncf/kubernetes/provider.yaml:
##
@@ -350,6 +350,15 @@ config:
type: string
example: ~
default: ""
+ task_publish_
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1477173486
##
airflow/providers/cncf/kubernetes/provider.yaml:
##
@@ -350,6 +350,15 @@ config:
type: string
example: ~
default: ""
+ task_publish_
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1477173614
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,9 +434,9 @@ def sync(self) -> None:
)
self.
dirrao commented on code in PR #36882:
URL: https://github.com/apache/airflow/pull/36882#discussion_r1477173515
##
airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:
##
@@ -434,19 +438,35 @@ def sync(self) -> None:
)
sel
dirrao commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1925609810
@potiuk / @hussein-awala
PR unrelated celery executor test is failing. is this due to recent change?
`tests/integration/executors/test_celery_executor.py::TestCeleryExecutor::test_
dirrao commented on PR #36882:
URL: https://github.com/apache/airflow/pull/36882#issuecomment-1932418167
@hussein-awala Can you re-review it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
potiuk merged PR #36882:
URL: https://github.com/apache/airflow/pull/36882
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.a
48 matches
Mail list logo