kacpermuda commented on code in PR #54807: URL: https://github.com/apache/airflow/pull/54807#discussion_r2318431508
########## providers/openlineage/tests/system/openlineage/operator.py: ########## Review Comment: Hey, thanks for the input. The operator itself is only used in system tests and is not publicly available for the users, it's really just reading OL events, emitted by test DAGs, from Airflow Variables and comparing them against expected values. The problem is that sometimes, there are many DAGs writing to Airflow Variables at the same time, and as the operator runs just after the task it's supposed to check, the variable may not be updated yet, so the whole check crashes with Error: Variable does not exist. The system tests do not run in Airflow repo automatically, users or companies can choose to run them on their own if they want, so there should be no major impact after that change. Anyone can always modify the code when running their tests, to make sure it suits their needs. I understand that use of time sleep may be problematic, but I think since those tests are run manually anyway, we should be fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
