potiuk commented on issue #27069: URL: https://github.com/apache/airflow/issues/27069#issuecomment-1288244591
BTW. In this particular case, i think the problem is that either test does not test what it should, or maybe the hearbeat mechanism should be changed to better match expectations. The fact that hearbeats come wiht significant delays on a heavily loaded system if we expect them to come regularly, is actually disconnect between the expectations and reality. We should either have more relaxed expectations of the heartbeat frequency (for example accept and communicate that heartbeats coming with 100% delay are still ok) or make sure heartbeats are generated with very good regularity independent from the load of the system. I think regularity Is hard to achieve in case of Python due to mutli-processing /GIL and the fact that heartbeats use database while generating heartbeats - this combination make it nearly impossible to guarantee that heartbeat will come with great regularity. So I'd s relaxing it and looking at what is the consequence of it is likely better way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
