Hi All,
We (owlabs - fork: https://github.com/owlabs/incubator-airflow) have a high
level design for how to improve the logging throughout the Airflow code to be
more consistent, maintainable and extensible. We'd really appreciate any
feedback on the design.
Design for Consolidating Logging Setup:
* Create a Class in the utils\logging.py to instantiate and handle setup of
an "airflow" logger object. This allows us to easily find all setup code for
further modification and extension in the future.
* This class would be where any general logging configuration (from
Airflow.cfg) would be applied.
* Instantiate this class in this file, so that importing it allows easy
control of the logging setup from anywhere else in the application.
* Move all setup of the logging in general to this class with easy to call
control methods to turn forms of logging on and off (e.g.
console/debugFile/errorFile).
* Move any other helper functions related to logging that are still required
into the utils\logging.py so that we can easily find them for modification and
extension in the future.
* Ensure that all logging throughout the application is done via the
"airflow" logger object, or by a child of this logger object. This allows us to:
* Leave all settings of the root logger object alone, so that we are
friendly to any parent or child processes that are not part of the application,
allowing them to safely manage their own overriding logging setup.
* Modify the settings of logging throughout the entire application via a
single simple point of control.
Cheers,
Luke Maycock
OLIVER WYMAN
luke.mayc...@affiliate.oliverwyman.com<mailto:luke.mayc...@affiliate.oliverwyman.com>
www.oliverwyman.com<http://www.oliverwyman.com/>
________________________________
This e-mail and any attachments may be confidential or legally privileged. If
you received this message in error or are not the intended recipient, you
should destroy the e-mail message and any attachments or copies, and you are
prohibited from retaining, distributing, disclosing or using any information
contained herein. Please inform us of the erroneous delivery by return e-mail.
Thank you for your cooperation.
