crabio opened a new issue, #36478:
URL: https://github.com/apache/airflow/issues/36478
### Apache Airflow version
2.8.0
### If "Other Airflow 2 version" selected, which one?
2.7.3
### What happened?
I found race condition between 2 schedulers.
Schedul
crabio opened a new issue, #36478:
URL: https://github.com/apache/airflow/issues/36478
### Apache Airflow version
2.8.0
### If "Other Airflow 2 version" selected, which one?
2.7.3
### What happened?
I found race condition between 2 schedulers.
Schedul
crabio opened a new issue, #35354:
URL: https://github.com/apache/airflow/issues/35354
### Apache Airflow version
2.7.2
### What happened
1. Airflow Scheduler has periodic errors:
```log
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3
crabio commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1871513362
Discussed in https://github.com/apache/airflow/discussions/35426
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
crabio commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1872228151
Thank you (@dirrao) very much!
I will try and be back soon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and us
crabio commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1873982378
@dirrao Seems it works well! thank you
I will watch for some days for fix and reopen issue if anything will be wrong
--
This is an automated message from the Apache Git Service.
crabio closed issue #36478: Airflow Scheduler with Kubernetes Executor has
errors in logs and stuck slots with no running tasks
URL: https://github.com/apache/airflow/issues/36478
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
crabio commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1878294865
Hello @dirrao
I upgraded my Airflow package
`apache-airflow-providers-cncf-kubernetes==7.13.0` and increased Schedulers
replicas to 2 and in our Dev environment all was fine. Bu
dirrao commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1878329845
@crabio,
Thanks for updating us on this. This requires triaging. Meanwhile, you can
bump up the parallelism configuration to a higher number to beat the leak. Or
Restart the sche
crabio commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1878480186
@dirrao Yes, thank you!
I rollback our cluster to 1 Scheduler and 64 parallelism to have the same
capacity :)
About restart - I found that it could be made in the Airflow <
dirrao commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1884755897
> @dirrao Yes, thank you!
>
> I rollback our cluster to 1 Scheduler and 64 parallelism to have the same
capacity :) About restart - I found that it could be made in the Airflo
crabio commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-2044294535
duplicated in https://github.com/apache/airflow/issues/36998
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
github-actions[bot] commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1909132579
This issue has been automatically marked as stale because it has been open
for 14 days with no response from the author. It will be closed in next 7 days
if no further
github-actions[bot] commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-1922529692
This issue has been closed because it has not received response from the
issue author.
--
This is an automated message from the Apache Git Service.
To respond to the
github-actions[bot] closed issue #36478: Airflow Scheduler with Kubernetes
Executor has errors in logs and stuck slots with no running tasks
URL: https://github.com/apache/airflow/issues/36478
--
This is an automated message from the Apache Git Service.
To respond to the message, please log o
ephraimbuddy commented on issue #36478:
URL: https://github.com/apache/airflow/issues/36478#issuecomment-2141737200
duplicated in https://github.com/apache/airflow/issues/36998
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
boring-cyborg[bot] commented on issue #35354:
URL: https://github.com/apache/airflow/issues/35354#issuecomment-1789335981
Thanks for opening your first issue here! Be sure to follow the issue
template! If you are willing to raise PR to address this issue please do so, no
need to wait for ap
crabio commented on issue #35354:
URL: https://github.com/apache/airflow/issues/35354#issuecomment-1790365264
Also we figured out that some time tasks failed without any reason and
without retries.
```log
[[34m2023-11-02T03:13:05.686+[0m] {[34mtaskinstance.py:[0m1441}
WARN
jens-scheffler-bosch commented on issue #35354:
URL: https://github.com/apache/airflow/issues/35354#issuecomment-1791414114
Your descriptions are a bit vague but I can say that the scheduler does not
create empty slots in advance with KubernetesExecutor. For each task being
executed, one PO
crabio commented on issue #35354:
URL: https://github.com/apache/airflow/issues/35354#issuecomment-1792920774
In my case I see that Airflow Scheduler after some time not queuing tasks.
But we have empty slots in pool and all components are running.
No pods are spawned. All tasks are f
crabio commented on issue #35354:
URL: https://github.com/apache/airflow/issues/35354#issuecomment-1792924862
Maybe it is same issue:
Some times tasks is running well after Scheduling restart. But random task
can fail without any logs or attempts.
If I clear it state via UI - task
jens-scheffler-bosch commented on issue #35354:
URL: https://github.com/apache/airflow/issues/35354#issuecomment-1793144154
This rather sounds like troubleshooting.
I have never used KubernetesExecutor myself, so unfortunately I have not
further experience and would assume that Scheduler
jens-scheffler-bosch closed issue #35354: Airflow Scheduler with Kubernetes
Executor has errors in logs and stuck slots with no running tasks
URL: https://github.com/apache/airflow/issues/35354
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
23 matches
Mail list logo
