Daniel Standish created AIRFLOW-4134: ----------------------------------------
Summary: DB connection invalidated warning at every zombie check Key: AIRFLOW-4134 URL: https://issues.apache.org/jira/browse/AIRFLOW-4134 Project: Apache Airflow Issue Type: Bug Affects Versions: 1.10.2 Reporter: Daniel Standish I am finding with 1.10.2 that I seem to get a warning {{DB connection invalidated. Reconnecting...}} very frequently. I to try to diagnose I added logging of the triggering error on line 79 in airflow/utils/sqlalchemy.py, from which this warning is generated. Here's the traceback: {code} webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/utils/sqlalchemy.py", line 68, in ping_connection webserver_1 | connection.scalar(select([1])) webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 912, in scalar webserver_1 | return self.execute(object_, *multiparams, **params).scalar() webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 980, in execute webserver_1 | return meth(self, multiparams, params) webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection webserver_1 | return connection._execute_clauseelement(self, multiparams, params) webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement webserver_1 | distilled_params, webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context webserver_1 | e, statement, parameters, cursor, context webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1458, in _handle_dbapi_exception webserver_1 | util.raise_from_cause(sqlalchemy_exception, exc_info) webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause webserver_1 | reraise(type(exception), exception, tb=exc_tb, cause=cause) webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 276, in reraise webserver_1 | raise value.with_traceback(tb) webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context webserver_1 | cursor, statement, parameters, context webserver_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute webserver_1 | cursor.execute(statement, parameters) webserver_1 | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly webserver_1 | This probably means the server terminated abnormally webserver_1 | before or while processing the request. webserver_1 | [SQL: 'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8) {code} I am not sure where to go from here. But something must be wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)