Good point Chris. I overlooked that. B.
> On 16 Jan 2018, at 20:09, Chris Palmer <crisp...@gmail.com> wrote: > > I'm not sure this is the right solution. I haven't explored all the code > but it seems to me that the actual database connections are managed in the > hooks. Different databases will want to handle this differently, and > modifying SqlSensor seems too heavy handed to me. > > > If you look at the get_records method of the generic DbApiHook ( > https://pythonhosted.org/airflow/_modules/dbapi_hook.html#DbApiHook) then > you can see it does closer the connections it is making. > > Is there specific hook for Teradata, and have you looked at how that is > handling connections? > > Chris > > > On Jan 16, 2018 9:32 AM, "Bolke de Bruin" <bdbr...@gmail.com> wrote: > > Of course. PR welcome. It would be nice to know how to test for connection > leakage. > > Sent from my iPhone > >> On 16 Jan 2018, at 16:01, Alexis Rolland <alexis.roll...@ubisoft.com> > wrote: >> >> Hello everyone, >> >> I’m reaching out to discuss / suggest a small improvement in the class > SqlSensor: >> https://pythonhosted.org/airflow/_modules/airflow/operators/sensors.html >> >> We are currently using SqlSensors on top of Teradata in several DAGs. > When the DAGs execute we receive the following error message from Teradata > engine: Error 8024 All virtual circuits are currently in use. >> This error message would typically appear when we reach the maximum > number of simultaneous connections to the database. >> >> I am suspecting the SqlSensor task to create a lot of connections - > basically every time it (re)tries, and these connections would end up in > idle state. >> Does closing the connection at the end of the SqlSensor poke method > sounds feasible? >> >> I’d like to take this opportunity as well to thank you for the awesome > work you’ve been doing with Airflow. >> Keep it up! >> >> Best, >> >> >>
