[
https://issues.apache.org/jira/browse/AIRFLOW-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888692#comment-16888692
]
Mikołaj Morawski commented on AIRFLOW-4593:
-------------------------------------------
I found only 31424:
{code:java}
[2019-07-19 08:11:15,607] {{logging_mixin.py:95}} INFO - [2019-07-19
08:11:15,607] {{settings.py:182}} INFO - settings.configure_orm(): Using pool
settings. pool_size=5, pool_recycle=1800, pid=31424
[2019-07-19 08:11:15,610] {{jobs.py:398}} INFO - Started process (PID=31424) to
work on /usr/local/airflow/dags/validation_dag.py
[2019-07-19 08:11:15,611] {{jobs.py:1711}} INFO - Processing file
/usr/local/airflow/dags/validation_dag.py for tasks to queue
[2019-07-19 08:11:15,612] {{logging_mixin.py:95}} INFO - [2019-07-19
08:11:15,612] {{__init__.py:305}} INFO - Filling up the DagBag from
/usr/local/airflow/dags/validation_dag.py
[2019-07-19 08:11:15,729] {{jobs.py:1723}} INFO - DAG(s)
dict_keys(['validation_dag']) retrieved from
/usr/local/airflow/dags/validation_dag.py
[2019-07-19 08:11:15,740] {{jobs.py:1446}} INFO - Processing validation_dag
[2019-07-19 08:11:15,745] {{jobs.py:632}} INFO - Skipping SLA check for <DAG:
validation_dag> because no tasks in DAG have SLAs
[2019-07-19 08:11:15,747] {{jobs.py:406}} INFO - Processing
/usr/local/airflow/dags/validation_dag.py took 0.138 seconds
[2019-07-19 08:11:21,615] {{logging_mixin.py:95}} INFO - [2019-07-19
08:11:21,615] {{settings.py:182}} INFO - settings.configure_orm(): Using pool
settings. pool_size=5, pool_recycle=1800, pid=31446{code}
This log is from scheduler logs. The 31425 was not found. After this PID log a
new PID is spawning in next 5 seconds.
I agree with you that this is interesting that memory is not returned to the OS
after docker restart. Even docker-engine restart does not help. I need to
reboot whole machine.That is why i am wondering if this in related to Docker.
Additionally: Do you think that i can downgrade Airflow to check this issue. To
which version ?
> Memory leak in Airflow scheduler
> --------------------------------
>
> Key: AIRFLOW-4593
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4593
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 1.10.2
> Reporter: Nikhil SInghal
> Priority: Major
> Attachments: Screenshot 2019-05-30 at 3.19.33 PM.png
>
>
> We are running Apache Airflow on Kubernetes. When I see my Grafana Dashboard
> I see that the memory used is consistently increasing. Can anyone give me
> some pointers for how can I debug this or with existing issues/solutions
> related to this.
> This is a sharp increase in Airflow scheduler and slow increase for worker
> and webserver
> !Screenshot 2019-05-30 at 3.19.33 PM.png|width=100%!
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
