[jira] [Commented] (AIRFLOW-6010) Remove cyclic imports and pylint hacks
[ https://issues.apache.org/jira/browse/AIRFLOW-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999403#comment-16999403 ] Kaxil Naik commented on AIRFLOW-6010: - Note: DAG Serialization bits have been cherry-picked to v1-10-tests https://github.com/apache/airflow/commit/8e82b2dcab01776f1562ea26cdf6565eca55ceda > Remove cyclic imports and pylint hacks > -- > > Key: AIRFLOW-6010 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6010 > Project: Apache Airflow > Issue Type: Sub-task > Components: core >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Assignee: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > > [AIRFLOW-6010] Remove cyclic imports and pylint hacks > There were a number of problems involving cyclic imports in > Airflow's core. Mainly about settingsi, DAG context management, base operator > imports and serialisation. > Some of those problems were workarounded by #pylint: disables (for pylint), > some of them were bypassed with TYPE_CHECKING (for mypy) and some of them > were > just hidden because pylint check was splitting filei lists while TravisiCI > build. > This commit fixes most of the problems (only executor problem is left) and > removes all the workarounds. > The fixes are: > * Context for DAG context management was loaded from settings and > Now context managemen is moved to DAG and 'import settings' is not > needed in baseoperator, subdag_operator. > * Serialized Fields are lazy initialised - they were previously > initialized while parsing the python modules which made it impossible to > avoid > cycles. > * SerializedDagModel is removed from 'airflow.models' and imported > directly from serialization package. This is only internal class and does not > need to be exposed via models > * BaseOperator in core is imported from baseoperator package > rather than from 'airflow.models'. This helps in importing the whole airflow > __init__ of 'airflow' without having to pay attention > to the sequence of imports there. > * BaseOperator on the other hand is imported from 'airflowi.models' in > operators/DAGs/hooks/sensors. This is important for Backporting (AIP-21) > * The imports of BaseOperator are enforced with pre-commit. > * All the pylint/mypy hacks related to cyclic imports are removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6010) Remove cyclic imports and pylint hacks
[ https://issues.apache.org/jira/browse/AIRFLOW-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982959#comment-16982959 ] ASF subversion and git services commented on AIRFLOW-6010: -- Commit 03c870a6172ab232af6319a30ad8d46622359b10 in airflow's branch refs/heads/master from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=03c870a ] [AIRFLOW-6010] Remove cyclic imports and pylint hacks (#6601) > Remove cyclic imports and pylint hacks > -- > > Key: AIRFLOW-6010 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6010 > Project: Apache Airflow > Issue Type: Sub-task > Components: core >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > > [AIRFLOW-6010] Remove cyclic imports and pylint hacks > There were a number of problems involving cyclic imports in > Airflow's core. Mainly about settingsi, DAG context management, base operator > imports and serialisation. > Some of those problems were workarounded by #pylint: disables (for pylint), > some of them were bypassed with TYPE_CHECKING (for mypy) and some of them > were > just hidden because pylint check was splitting filei lists while TravisiCI > build. > This commit fixes most of the problems (only executor problem is left) and > removes all the workarounds. > The fixes are: > * Context for DAG context management was loaded from settings and > Now context managemen is moved to DAG and 'import settings' is not > needed in baseoperator, subdag_operator. > * Serialized Fields are lazy initialised - they were previously > initialized while parsing the python modules which made it impossible to > avoid > cycles. > * SerializedDagModel is removed from 'airflow.models' and imported > directly from serialization package. This is only internal class and does not > need to be exposed via models > * BaseOperator in core is imported from baseoperator package > rather than from 'airflow.models'. This helps in importing the whole airflow > __init__ of 'airflow' without having to pay attention > to the sequence of imports there. > * BaseOperator on the other hand is imported from 'airflowi.models' in > operators/DAGs/hooks/sensors. This is important for Backporting (AIP-21) > * The imports of BaseOperator are enforced with pre-commit. > * All the pylint/mypy hacks related to cyclic imports are removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6010) Remove cyclic imports and pylint hacks
[ https://issues.apache.org/jira/browse/AIRFLOW-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982958#comment-16982958 ] ASF GitHub Bot commented on AIRFLOW-6010: - kaxil commented on pull request #6601: [AIRFLOW-6010] Remove cyclic imports and pylint disables URL: https://github.com/apache/airflow/pull/6601 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Remove cyclic imports and pylint hacks > -- > > Key: AIRFLOW-6010 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6010 > Project: Apache Airflow > Issue Type: Sub-task > Components: core >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > > [AIRFLOW-6010] Remove cyclic imports and pylint hacks > There were a number of problems involving cyclic imports in > Airflow's core. Mainly about settingsi, DAG context management, base operator > imports and serialisation. > Some of those problems were workarounded by #pylint: disables (for pylint), > some of them were bypassed with TYPE_CHECKING (for mypy) and some of them > were > just hidden because pylint check was splitting filei lists while TravisiCI > build. > This commit fixes most of the problems (only executor problem is left) and > removes all the workarounds. > The fixes are: > * Context for DAG context management was loaded from settings and > Now context managemen is moved to DAG and 'import settings' is not > needed in baseoperator, subdag_operator. > * Serialized Fields are lazy initialised - they were previously > initialized while parsing the python modules which made it impossible to > avoid > cycles. > * SerializedDagModel is removed from 'airflow.models' and imported > directly from serialization package. This is only internal class and does not > need to be exposed via models > * BaseOperator in core is imported from baseoperator package > rather than from 'airflow.models'. This helps in importing the whole airflow > __init__ of 'airflow' without having to pay attention > to the sequence of imports there. > * BaseOperator on the other hand is imported from 'airflowi.models' in > operators/DAGs/hooks/sensors. This is important for Backporting (AIP-21) > * The imports of BaseOperator are enforced with pre-commit. > * All the pylint/mypy hacks related to cyclic imports are removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
