tirkarthi commented on issue #49887:
URL: https://github.com/apache/airflow/issues/49887#issuecomment-2868630859
@zachliu I was checking on this yesterday. Can you please try the below
patch? I tried the sample dag with a copy of 100 dags. The transport object
which is created per client could be causing the socket to be open is my
assumption as well. That's why this is prevalent in dags with connection,
variable etc. that triggers client creation and subsequently transport creation
which I feel could be cached.
```diff
diff --git a/airflow-core/src/airflow/dag_processing/processor.py
b/airflow-core/src/airflow/dag_processing/processor.py
index 73d2c23c7f..19dd3e018c 100644
--- a/airflow-core/src/airflow/dag_processing/processor.py
+++ b/airflow-core/src/airflow/dag_processing/processor.py
@@ -213,6 +213,7 @@ class DagFileParsingResult(BaseModel):
type: Literal["DagFileParsingResult"] = "DagFileParsingResult"
+@functools.cache
def in_process_api_server() -> InProcessExecutionAPI:
from airflow.api_fastapi.execution_api.app import InProcessExecutionAPI
```
```python
# 1.py
from airflow.sdk import Variable
from airflow.models.dag import DAG
email = Variable.get(
f"email",
default=["t...@email.com"],
deserialize_json=False,
)
```
100 copies of the same file in dags folder
```
seq 1 100 | xargs -I{} cp ~/airflow/dags/1.py ~/airflow/dags/{}.py
```
```
pgrep -f dag-processor | xargs -I{} ls -l /proc/{}/fd/ | grep -i socket
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
