Re: [D] Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? [airflow]
GitHub user smktpd closed the discussion with a comment: Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? I was not aware that there already were plans of getting away from cyclic dependencies. The existence of such plans basically answers the initial question. Thanks! GitHub link: https://github.com/apache/airflow/discussions/61474#discussioncomment-15909339 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? [airflow]
GitHub user potiuk closed the discussion with a comment: Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? Becaue your solution will not work outside of docker - 30% of our users do not use any containerisation - so any solutions based on docker will not help those users, so proposing it makes no sense. See https://airflow.apache.org/blog/airflow-survey-2025/ Cyclic dependencies (for now) is needed until we finish task-isolation work, Currently providers depend on `apache-airflow` but in airflow 3.2 or 3.3 - they are supposed to only depend on a new `task-sdk` distribution rather than on `apache-airflow`. Removing those cyclic dependencies was one of the goals of https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-72+Task+Execution+Interface+aka+Task+SDK which is still not completed. You can read more about history and reasoning there. GitHub link: https://github.com/apache/airflow/discussions/61474#discussioncomment-15829350 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? [airflow]
GitHub user smktpd closed the discussion with a comment: Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? There are multiple correct answers to this question, but the main reason is isolation via containerization, of course. Then docker became quite a popular platform with lots of systems already made by someone else (so you don't have to do it yourself). I don't understand what this question has to do with the topic. Why keep circular deps? Why not organize dependencies so that there would be no circular deps? GitHub link: https://github.com/apache/airflow/discussions/61474#discussioncomment-15824712 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? [airflow]
GitHub user potiuk closed the discussion with a comment: Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? Why do you think everyone uses docker ? GitHub link: https://github.com/apache/airflow/discussions/61474#discussioncomment-15808070 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? [airflow]
GitHub user smktpd edited a comment on the discussion: Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? Why have them pre-installed? Why not install them? Wouldn't it be better to avoid circular dependencies and have them listed separately in the dockerfile instructions that are used to produce your images? GitHub link: https://github.com/apache/airflow/discussions/61474#discussioncomment-15707028 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? [airflow]
GitHub user smktpd closed the discussion with a comment: Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? Why have them pre-installed? Why not install them? GitHub link: https://github.com/apache/airflow/discussions/61474#discussioncomment-15707028 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? [airflow]
GitHub user smktpd closed a discussion: Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? I use `pipdeptree` to watch dependency graphs of pip packages in my images. With image docker.io/apache/airflow:2.11.0-python3.12 it notifies me of such cyclic dependencies: ``` Warning!!! Cyclic dependencies found: * apache-airflow-providers-common-compat => apache-airflow => apache-airflow-providers-common-compat * apache-airflow-providers-common-io => apache-airflow => apache-airflow-providers-common-io * apache-airflow-providers-common-sql => apache-airflow => apache-airflow-providers-common-sql * apache-airflow-providers-fab => apache-airflow-providers-common-compat => apache-airflow => apache-airflow-providers-fab * apache-airflow-providers-ftp => apache-airflow => apache-airflow-providers-ftp * apache-airflow-providers-http => apache-airflow => apache-airflow-providers-http * apache-airflow-providers-imap => apache-airflow => apache-airflow-providers-imap * apache-airflow-providers-smtp => apache-airflow => apache-airflow-providers-smtp * apache-airflow-providers-sqlite => apache-airflow => apache-airflow-providers-sqlite * apache-airflow => apache-airflow-providers-common-compat => apache-airflow ``` I understand that this got to be non-critical, but still feels off, so I wonder why they are made so. GitHub link: https://github.com/apache/airflow/discussions/61474 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? [airflow]
GitHub user potiuk closed the discussion with a comment: Why do pip packages apache-airflow-providers-* and apache-airflow have cyclic dependencies? Because Cyclic dependencies are perfectly OK in Python resolution and this is the only way you can have pre-installed providers. GitHub link: https://github.com/apache/airflow/discussions/61474#discussioncomment-15701112 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
