[PR] openlineage: add `opt-in` option [airflow]

2024-02-26 Thread via GitHub
JDarDagran opened a new pull request, #37725: URL: https://github.com/apache/airflow/pull/37725 ### Problem statement Currently you can only disable OpenLineage for all DAGs and tasks or selectively disable by operator class. Users report lack of possibility to do

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-21 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1534099758 ## airflow/providers/openlineage/plugins/listener.py: ## @@ -51,6 +57,15 @@ def __init__(self): self.log = logging.getLogger(__name__) self.extracto

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-21 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1534099758 ## airflow/providers/openlineage/plugins/listener.py: ## @@ -51,6 +57,15 @@ def __init__(self): self.log = logging.getLogger(__name__) self.extracto

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-21 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1534104809 ## airflow/providers/openlineage/utils/selective_enable.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor l

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-21 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1534109411 ## airflow/providers/openlineage/utils/selective_enable.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor l

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-21 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1534113809 ## airflow/providers/openlineage/utils/selective_enable.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor l

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-22 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1535448401 ## airflow/providers/openlineage/utils/selective_enable.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributo

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-22 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1535471096 ## airflow/providers/openlineage/utils/selective_enable.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributo

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-22 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1535468703 ## airflow/providers/openlineage/utils/selective_enable.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributo

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-25 Thread via GitHub
kacpermuda commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1537810497 ## tests/providers/openlineage/utils/test_selective_enable.py: ## @@ -0,0 +1,72 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more cont

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-25 Thread via GitHub
kacpermuda commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1537812684 ## airflow/providers/openlineage/plugins/listener.py: ## @@ -51,6 +57,16 @@ def __init__(self): self.log = logging.getLogger(__name__) self.extra

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-25 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538317937 ## airflow/providers/openlineage/plugins/listener.py: ## @@ -51,6 +57,16 @@ def __init__(self): self.log = logging.getLogger(__name__) self.extra

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
uranusjr commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538855386 ## airflow/providers/openlineage/utils/utils.py: ## @@ -73,6 +78,16 @@ def is_operator_disabled(operator: BaseOperator | MappedOperator) -> bool: return get_fu

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
uranusjr commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538855386 ## airflow/providers/openlineage/utils/utils.py: ## @@ -73,6 +78,16 @@ def is_operator_disabled(operator: BaseOperator | MappedOperator) -> bool: return get_fu

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
uranusjr commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538861208 ## airflow/providers/openlineage/utils/utils.py: ## @@ -73,6 +78,16 @@ def is_operator_disabled(operator: BaseOperator | MappedOperator) -> bool: return get_fu

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
uranusjr commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538862026 ## docs/apache-airflow-providers-openlineage/guides/user.rst: ## @@ -263,6 +264,61 @@ a string of semicolon separated Airflow Operators full import paths to ``extra

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538912119 ## docs/apache-airflow-providers-openlineage/guides/user.rst: ## @@ -263,6 +264,61 @@ a string of semicolon separated Airflow Operators full import paths to ``ext

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538912359 ## airflow/providers/openlineage/utils/utils.py: ## @@ -73,6 +78,16 @@ def is_operator_disabled(operator: BaseOperator | MappedOperator) -> bool: return get_

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
kacpermuda commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538934144 ## airflow/providers/openlineage/conf.py: ## @@ -51,6 +51,11 @@ def disabled_operators() -> set[str]: return set(operator.strip() for operator in option.split

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1538954635 ## airflow/providers/openlineage/conf.py: ## @@ -51,6 +51,11 @@ def disabled_operators() -> set[str]: return set(operator.strip() for operator in option.split

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-26 Thread via GitHub
mobuchowski merged PR #37725: URL: https://github.com/apache/airflow/pull/37725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-02-27 Thread via GitHub
mobuchowski commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1503988848 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyt

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-02-27 Thread via GitHub
uranusjr commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1504037856 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Python

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-02-27 Thread via GitHub
kacpermuda commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1504062094 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyth

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-02-27 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1504068148 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyth

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-02-27 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1504070137 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyth

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-02-27 Thread via GitHub
kacpermuda commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1504081457 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyth

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-02-27 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1504123209 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyth

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-02-28 Thread via GitHub
mobuchowski commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1505708132 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyt

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-08 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1517880761 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.PythonO

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-08 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1517880761 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.PythonO

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-08 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1517880761 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.PythonO

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-08 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1517892095 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.PythonO

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-08 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1517942728 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyth

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-08 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1517976751 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.PythonO

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-08 Thread via GitHub
mobuchowski commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1517981734 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyt

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-08 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1517976751 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.PythonO

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-11 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1519514121 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.PythonO

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-11 Thread via GitHub
tatiana commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1519514121 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.PythonO

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-11 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1519550481 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyth

Re: [PR] openlineage: add `opt-in` option [airflow]

2024-03-11 Thread via GitHub
JDarDagran commented on code in PR #37725: URL: https://github.com/apache/airflow/pull/37725#discussion_r1519550694 ## airflow/providers/openlineage/provider.yaml: ## @@ -77,6 +77,14 @@ config: example: "airflow.operators.bash.BashOperator;airflow.operators.python.Pyth