kaxil commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r350381674
########## File path: airflow/lineage/__init__.py ########## @@ -16,53 +16,85 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. - -# pylint:disable=missing-docstring - +""" +Provides lineage support functions +""" +import json from functools import wraps -from itertools import chain +from typing import Any, Dict, Optional -from airflow.configuration import conf -from airflow.exceptions import AirflowConfigException -from airflow.lineage.datasets import DataSet +import attr +import jinja2 +from cattr import structure, unstructure + +from airflow.models.base import Operator from airflow.utils.log.logging_mixin import LoggingMixin from airflow.utils.module_loading import import_string +ENV = jinja2.Environment() + PIPELINE_OUTLETS = "pipeline_outlets" PIPELINE_INLETS = "pipeline_inlets" +AUTO = "auto" log = LoggingMixin().log -def _get_backend(): - backend = None [email protected](auto_attribs=True) +class Metadata: + """ + Class for serialized entities. + """ + type: str = attr.ib() Review comment: Didn't knew about `attr` library, looks powerful. Thanks for it Should we rename `type` to `_type` or `metadata_type` so that we don't use a built-in keyword ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
