Re: SqlAlchemy Pool config parameters to minimize connectivity issue impact

2018-10-14 Thread Pramiti Goel
Hi, We also faced this issue one month back for Airflow 1.9 with Celery Executor. Unfortunately, we could not find the root cause immediately. This can occur due to too many open connections. We end up closing all open connections as immediate Solution and restarting Mysql Instance. But I think

Re: SqlAlchemy Pool config parameters to minimize connectivity issue impact

2018-09-28 Thread raman gupta
Hi Kevin, We are using Airflow 1.9 with Local Excecutor and we are facing various mysql connectivity error/exceptions like -> sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') [SQL: u'UPDATE

SqlAlchemy Pool config parameters to minimize connectivity issue impact

2018-09-26 Thread ramandumcs
Hi All, We are observing sometimes Dag tasks get failed because of some connectivity issues with Mysql server. So Are there any recommended settings for mysql pool's related parameters like sql_alchemy_pool_size = 5 sql_alchemy_pool_recycle = 3600 to minimise the connectivity issue impact.