Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user potiuk added a comment to the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 And exact reproduction steps GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-14210783 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user potiuk added a comment to the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 Can you please open new issue with your log details and exact lines of error from latest version? GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-14210778 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user tomaszstepinvest added a comment to the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 pip install "apache-airflow==3.0.x" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.0.x/constraints-3.12.txt"; I did the same for 3.0.5, 3.0.4, 3.0.1 Exactly the same error GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-14210107 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user jtantry added a comment to the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 Is there a solution for this? I am facing the same issue for Airflow 3.0.2 and Python 3.12 version. GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-13630727 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user sehoffmann edited a comment on the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 I have the same issue and error. I can not get a simple dag to run without the scheduler crashing. I also get the same HTTP response when manually going to API url in my browser. GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-13129222 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user sehoffmann added a comment to the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 I have the same issue and error. I can not get a simple dag to run without the scheduler crashing. GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-13129222 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user fengchutadie added a comment to the discussion: When start a dag in
airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4
it seems it's not about out of resources.
in console log, part of the scheduler log as below
```
scheduler | [2025-05-13T01:58:37.100+] {_client.py:1026} INFO - HTTP
Request: PATCH
http://localhost:8080/execution/task-instances/0196c1ed-1bfe-7616-961a-af093a67ca90/run
"HTTP/1.1 504 Gateway Timeout"
scheduler | 2025-05-13 01:58:37 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 4th time calling it.
[airflow.sdk.api.client]
scheduler | [2025-05-13T01:58:38.897+] {_client.py:1026} INFO - HTTP
Request: PATCH
http://localhost:8080/execution/task-instances/0196c1ed-1bfe-7616-961a-af093a67ca90/run
"HTTP/1.1 504 Gateway Timeout"
scheduler | 2025-05-13 01:58:38 [info ] Process exited
[supervisor] exit_code= pid=3129186 signal_sent=SIGKILL
scheduler | [2025-05-13T01:58:38.907+] {local_executor.py:96} ERROR - uhoh
scheduler | httpx.HTTPStatusError: Server error '504 Gateway Timeout' for url
'http://localhost:8080/execution/task-instances/0196c1ed-1bfe-7616-961a-af093a67ca90/run'
scheduler | For more information check:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/504
scheduler | Correlation-id=0196c75d-8bf5-75ca-bd0b-365e12de83ce
scheduler | [2025-05-13T01:58:39.917+] {scheduler_job_runner.py:1019}
ERROR - Exception when executing SchedulerJob._run_scheduler_loop
scheduler | TypeError: HTTPStatusError.__init__() missing 2 required
keyword-only arguments: 'request' and 'response'
scheduler | [2025-05-13T01:58:39.921+] {local_executor.py:223} INFO -
Shutting down LocalExecutor; waiting for running tasks to finish. Signal again
if you don't want to wait.
scheduler | [2025-05-13T01:58:39.935+] {scheduler_job_runner.py:1031} INFO
- Exited execute loop
scheduler | TypeError: HTTPStatusError.__init__() missing 2 required
keyword-only arguments: 'request' and 'response'
scheduler | [2025-05-13 01:58:40 +] [3129173] [INFO] Shutting down: Master
```
after serveral times of http request:
**HTTP Request: PATCH
http://localhost:8080/execution/task-instances/0196c1ed-1bfe-7616-961a-af093a67ca90/run
"HTTP/1.1 504 Gateway Timeout"**, then it would kill scheduler proactively as
log
```
scheduler | 2025-05-13 01:58:38 [info] Process exited [supervisor]
exit_code= pid=3129186 signal_sent=SIGKILL
```
in the host Linux, when I execute
**wget
http://localhost:8080/execution/task-instances/0196c1ed-1bfe-7616-961a-af093a67ca90/run**
it prints
```
--2025-05-13 02:02:35--
http://localhost:8080/execution/task-instances/0196c1ed-1bfe-7616-961a-af093a67ca90/run
Connecting to 10.158.100.9:8080... connected.
Proxy request sent, awaiting response... 504 Gateway Timeout
Retrying.
```
and when I get it in another host with browser
http://**10.57.200.121**:8080/execution/task-instances/0196c1ed-1bfe-7616-961a-af093a67ca90/run,
it shows
`{
"detail": "Method Not Allowed"
}`
it's not gateway timeout if I replace localhost by 10.57.200.121.
in airflow 2.10.4, it works well and processes in backend are different with
processes in airflow 3.0.1rc1, I'm not sure if the difference about process
communication mechanism causes this issue.
GitHub link:
https://github.com/apache/airflow/discussions/50382#discussioncomment-13125479
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user potiuk added a comment to the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 Maybe you simply run out of resources- memory for example. It's impossible to tell what problem you have without you digging deeper and providin more information. GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-13120007 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user fengchutadie added a comment to the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 I just prepare the whole env for version 3.0.1rc1. it seems a little different. - 3.0.0, when I trigger the example dag (tutorial_dag.py), the "airflow scheduler" would be defunct in several seconds. - 3.0.1rc1, trigger the same dag, it would be in queued status about several minutes and then "airflow scheduler" would be defunct. but they encounter the same issue, I still can't schedule any dag. GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-13111343 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 [airflow]
GitHub user potiuk added a comment to the discussion: When start a dag in airflow 3.0.0, scheduler could be crashed but works well in airflow 2.10.4 Please try 3.0.1rc1 -> it's currently being tested and I think there was a case fixed where API calls could go into infinite loop that would resilut in similar issue. GitHub link: https://github.com/apache/airflow/discussions/50382#discussioncomment-13086333 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
