BTW. We probably (and feel free :) could make it work when I think of it. currently --use-installation-from-dist works in two steps:.
a) installs specified airflow (or airflow from dist if found in dist) in a single operation without constraints b) installs all other distributions without constraints We could probably make it three step and handle this edge case by installing airflow separately (with constraints) then providers and all other things I will leave it as exercise for the readers though - the logic there is already very complex and handles quite a few other cases and this one might break some of them. :) śr., 5 lis 2025, 10:57 użytkownik Jarek Potiuk <[email protected]> napisał: > The problem is `--use-distributions-from-dist`. The > `--use-airflow-version` and --use-distributions-from-dist does not use the > `--use-airflow-version` constraints because it will then limit whatever is > instlalled from dist (constraints from 3.1.1 have older version of edge > provider and if you try to install new edge provider from "dist" and > constraints pointing to the 3.1.1 provider they will conflict. > > This is our classic installation pattern where you install airflow + > whatever was released with constraints and **then** in a separate step you > install any updates. Using both constraints and something that deviates > from them in the same command is not going to work. > > So the right way of installing the provider for old versions is: > > * breeze start-airflow --use-airflow-version 3.1.1 -> uses constraints > * install new provider in terminal inside > * restart the components in other tmux terminals > > The --use-distributions-from-dist also works but by the fact that we want > to install all the distributions together (including one that deviates from > original constraints) - we cannot use constraints for it. > > J. > > > > On Tue, Nov 4, 2025 at 11:09 PM Jens Scheffler <[email protected]> > wrote: > >> Hi Jarek, >> >> what I actually did to be preceise: >> >> 1) Checked out to tag "providers-edge3/1.4.1rc2" >> >> 2) Build edge3 and docker on that tag (I need docker as well for an >> extended test...) via >> breeze release-management prepare-provider-distributions edge3 docker >> --skip-tag-check >> >> 3) Checkout (latest) main to ensure we use feshest breeze >> >> 4) Start system via breeze using: >> breeze start-airflow --python 3.12 --load-example-dags --backend >> postgres --executor EdgeExecutor --answer y --use-airflow-version 3. >> 1.1 --use-distributions-from-dist >> >> Re tested tonight again, results: >> >> * 3.0.6 - OK >> * 3.1.0 - not compatible with edge3 1.4.1 :-D >> * 3.1.1 - fails in structlog >> * 3.1.2rc1 - OK >> * 3.1.2rc2 - OK >> >> Made no manual install. But as 3.1.2 is OK I am okay anyway. >> >> On 11/4/25 15:36, Jarek Potiuk wrote: >> >> Don't know why I catched the "wrong" constraints" via installing with >> > breeze - might be constraints are not considered when installed a >> specific >> > airflow version and then providers are upgraded? >> > >> > It could be that later when you upgraded some providers, this also get >> > bumped or maybe some uv sync in the image happened. When I used >> > `--use-airflow-version 3.1.1` and **then** installed just >> > `apache-airflow-providers-edge3==1.4.1rc2` in breeze - everything worked >> > fine. It also can be you have some customisation scripts in breeze that >> do >> > some upgrades afterwards and you have not noticed. >> > >> > On Tue, Nov 4, 2025 at 3:34 PM Jarek Potiuk<[email protected]> wrote: >> > >> >> It's actually already fixed in main and cherry-picked to 3.1.2 - > so >> in >> >> 3.1.2 it's OK to have structlog 25.5.0. See >> >> https://github.com/apache/airflow/pull/57335 and it's been >> cherry-picked >> >> and merged as part ofhttps://github.com/apache/airflow/pull/57640 >> >> >> >> J, >> >> >> >> >> >> On Tue, Nov 4, 2025 at 3:04 PM Scheffler Jens (XC-AS/EAE-ADA-T) via >> dev < >> >> [email protected]> wrote: >> >> >> >>> Hi Amough, >> >>> >> >>> you are right but constraints in Airflow 3.1.2rc[1|2] both pin >> >>> structlog==25.5.0 >> >>> see >> >>> >> https://raw.githubusercontent.com/apache/airflow/constraints-3.1.2rc2/constraints-3.10.txt >> >>> >> >>> So wanted to raise this if not detected that we need to fix >> constraints >> >>> before releasing 3.1.2 as well. >> >>> Don't know why I catched the "wrong" constraints" via installing with >> >>> breeze - might be constraints are not considered when installed a >> specific >> >>> airflow version and then providers are upgraded? >> >>> >> >>> Jens >> >>> >> >>> -----Original Message----- >> >>> From: Amogh Desai<[email protected]> >> >>> Sent: Dienstag, 4. November 2025 10:07 >> >>> To:[email protected] >> >>> Subject: Re: [VOTE] Airflow Providers prepared on November 03, 2025 >> >>> >> >>> Jens, we have constrained structlog to disallow 25.5.0: >> >>> >> >>> >> https://raw.githubusercontent.com/apache/airflow/constraints-3.1.0/constraints-3.10.txt >> >>> >> >>> So, that should not be a problem for you :), maybe it was installed in >> >>> your environment. >> >>> >> >>> Thanks & Regards, >> >>> Amogh Desai >> >>> >> >>> >> >>> On Tue, Nov 4, 2025 at 3:59 AM Jens Scheffler<[email protected]> >> >>> wrote: >> >>> >> >>>> +1 (binding) - Checked SVN, Check in Docker, Reproducible package >> >>>> +build, >> >>>> Licenses, Signatures >> >>>> >> >>>> Installed the edge provider with current common-compat in Airflow >> >>>> 2.11.0, 3.0.6, 3.1.2rc1 and main and tested with EdgeExecutor and >> >>>> Integration Test Dag, all loking good. >> >>>> >> >>>> Will test detailled issue resolution (subpath etc) tomorrow but until >> >>>> now all good to release. Best release we had for Edge3 ever :-D >> >>>> >> >>>> Note: Was not able to install providers with Airflow 3.1.1 as >> >>>> structlog==25.5.0 installed by breeze has some incompatability and >> >>>> breze start failed with - assume this is not blocking providers >> >>>> release - assume this is because constraints are installed from main >> >>>> and not 3.1.1 in breeze: >> >>>> >> >>>> Traceback (most recent call last): >> >>>> File "/usr/python/bin/airflow", line 4, in <module> >> >>>> from airflow.__main__ import main >> >>>> File >> >>>> "/usr/python/lib/python3.12/site-packages/airflow/__init__.py", >> >>>> line 79, in <module> >> >>>> settings.initialize() >> >>>> File >> >>>> "/usr/python/lib/python3.12/site-packages/airflow/settings.py", >> >>>> line 698, in initialize >> >>>> LOGGING_CLASS_PATH = configure_logging() >> >>>> ^^^^^^^^^^^^^^^^^^^ >> >>>> File >> >>>> "/usr/python/lib/python3.12/site-packages/airflow/logging_config.py", >> >>>> line 88, in configure_logging >> >>>> from airflow._shared.logging import configure_logging, >> >>>> init_log_folder, translate_config_values >> >>>> File >> >>>> >> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/__in >> >>>> it__.py", >> >>>> >> >>>> line 27, in <module> >> >>>> from .structlog import configure_logging, init_log_file, >> >>>> init_log_folder >> >>>> File >> >>>> >> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/stru >> >>>> ctlog.py", >> >>>> >> >>>> line 38, in <module> >> >>>> from .percent_formatter import PercentFormatRender >> >>>> File >> >>>> >> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/perc >> >>>> ent_formatter.py", >> >>>> >> >>>> line 29, in <module> >> >>>> from structlog.dev import ConsoleRenderer, Styles >> >>>> ImportError: cannot import name 'Styles' from 'structlog.dev' >> >>>> (/usr/python/lib/python3.12/site-packages/structlog/dev.py) >> >>>> >> >>>> Error: check_environment returned 1. Exiting. >> >>>> >> >>>> Call that I used for testing: breeze start-airflow --python 3.12 >> >>>> --load-example-dags --backend postgres --executor EdgeExecutor >> >>>> --answer y --use-airflow-version 3.1.1 --use-distributions-from-dist >> >>>> >> >>>> Tested also with Airflow 3.1.0 and saw as intended that edge3 >> provider >> >>>> 1.4.1rc2 can not installed with Airflow 3.1.0 as UI plugin >> incompatible. >> >>>> >> >>>> Using Python 3.12.12 environment at: /usr/python >> >>>> × No solution found when resolving dependencies: >> >>>> ╰─▶ Because only apache-airflow-providers-edge3==1.4.1 is >> available >> >>>> and apache-airflow-providers-edge3==1.4.1 depends on one of: >> >>>> apache-airflow>=2.10.0,<3.1.0 >> >>>> apache-airflow>3.1.0 >> >>>> we can conclude that all versions of >> >>>> apache-airflow-providers-edge3 depend on one of: >> >>>> apache-airflow>=2.10.0,<3.1.0 >> >>>> apache-airflow>3.1.0 >> >>>> >> >>>> And because you require apache-airflow==3.1.0 and >> >>>> apache-airflow-providers-edge3, we can conclude that your >> requirements >> >>>> are unsatisfiable. >> >>>> >> >>>> On 11/3/25 13:06, Elad Kalif wrote: >> >>>>> Hey all, >> >>>>> >> >>>>> I have just cut the ad-hoc wave Airflow Providers packages. This >> >>>>> email is calling a vote on the release, which will last for 72 hours >> >>>>> - which means that it will end on November 06, 2025 12:05 PM UTC and >> >>>>> until 3 binding +1 votes have been received. >> >>>>> >> >>>>> Consider this my (binding) +1. >> >>>>> >> >>>>> Airflow Providers are available at: >> >>>>> https://di/ >> >>>>> st.apache.org >> %2Frepos%2Fdist%2Fdev%2Fairflow%2Fproviders%2F&data=05% >> >>>>> 7C02%7CJens.Scheffler%40de.bosch.com >> %7Cf6343926cc624857a11208de1b819 >> >>>>> e4d%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681481189% >> >>>>> 7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs >> >>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata= >> >>>>> u%2BGuEPt6aFORkaSjATUG2TVoDJiUJmNDiTM9Txe4cQc%3D&reserved=0 >> >>>>> >> >>>>> *apache-airflow-providers-<PROVIDER>-*.tar.gz* are the binary >> >>>>> Python "sdist" release - they are also official "sources" for the >> >>>> Provider >> >>>>> distributions. >> >>>>> >> >>>>> *apache_airflow_providers_<PROVIDER>-*.whl are the binary >> >>>>> Python "wheel" release. >> >>>>> >> >>>>> The test procedure for PMC members is described in >> >>>>> >> >>>> https://gith/ >> >>>> ub.com >> %2Fapache%2Fairflow%2Fblob%2Fmain%2Fdev%2FREADME_RELEASE_PROVIDE >> >>>> >> RS.md%23verify-the-release-candidate-by-pmc-members&data=05%7C02%7CJen >> >>>> s.Scheffler%40de.bosch.com >> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51e >> >>>> >> 1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681516854%7CUnknown%7CTW >> >>>> >> FpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIs >> >>>> >> IkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6sgJ88Z0t%2F1Qnhxr >> >>>> wPTnej7SfgLNf6hI7qYxnJBl6RU%3D&reserved=0 >> >>>>> The test procedure for and Contributors who would like to test this >> >>>>> RC is described in: >> >>>>> >> >>>> https://gith/ >> >>>> ub.com >> %2Fapache%2Fairflow%2Fblob%2Fmain%2Fdev%2FREADME_RELEASE_PROVIDE >> >>>> >> RS.md%23verify-the-release-candidate-by-contributors&data=05%7C02%7CJe >> >>>> ns.Scheffler%40de.bosch.com >> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51 >> >>>> >> e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681546634%7CUnknown%7CT >> >>>> >> WFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiI >> >>>> >> sIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=j%2B4anx9P7T5uTdc >> >>>> omIOr%2BOr2%2FoDHQnntn1Pvhn33BHs%3D&reserved=0 >> >>>>> >> >>>>> Public keys are available at: >> >>>>> https://di/ >> >>>>> st.apache.org >> %2Frepos%2Fdist%2Frelease%2Fairflow%2FKEYS&data=05%7C02 >> >>>>> %7CJens.Scheffler%40de.bosch.com >> %7Cf6343926cc624857a11208de1b819e4d% >> >>>>> 7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681574562%7CUn >> >>>>> known%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAi >> >>>>> OiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=enB4 >> >>>>> BNSldvybyXLUzIJIz5jE2tCnZ904nXCrgzoZp%2Bg%3D&reserved=0 >> >>>>> >> >>>>> Please vote accordingly: >> >>>>> >> >>>>> [ ] +1 approve >> >>>>> [ ] +0 no opinion >> >>>>> [ ] -1 disapprove with the reason >> >>>>> >> >>>>> Only votes from PMC members are binding, but members of the >> >>>>> community are encouraged to test the release and vote with >> >>> "(non-binding)". >> >>>>> Please note that the version number excludes the 'rcX' string. >> >>>>> This will allow us to rename the artifact without modifying the >> >>>>> artifact checksums when we actually release. >> >>>>> >> >>>>> The status of testing the providers by the community is kept here: >> >>>>> https://gi/ >> >>>>> thub.com >> %2Fapache%2Fairflow%2Fissues%2F57739&data=05%7C02%7CJens.Sch >> >>>>> effler%40de.bosch.com >> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51e190 >> >>>>> 7c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681602068%7CUnknown%7CTWF >> >>>>> pbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiI >> >>>>> sIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=PPDrnZgPIfBJWRp >> >>>>> 5VyEq1Ij2NGJXs9qK3b9mJA7Cwjk%3D&reserved=0 >> >>>>> >> >>>>> The issue is also the easiest way to see important PRs included in >> >>>>> the RC candidates. >> >>>>> Detailed changelog for the providers will be published in the >> >>>> documentation >> >>>>> after the >> >>>>> RC candidates are released. >> >>>>> >> >>>>> You can find the RC packages in PyPI following these links: >> >>>>> >> >>>>> https://py/ >> >>>>> pi.org >> %2Fproject%2Fapache-airflow-providers-edge3%2F1.4.1rc2%2F&data >> >>>>> =05%7C02%7CJens.Scheffler%40de.bosch.com >> %7Cf6343926cc624857a11208de1 >> >>>>> b819e4d%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681628 >> >>>>> 771%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw >> >>>>> MCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sd >> >>>>> ata=HWxblv2u1%2B6Pr3QrB5N1s9k7wnOcgLZNi7qsgW3t9i0%3D&reserved=0 >> >>>>> >> >>>>> Cheers, >> >>>>> Elad Kalif >> >>>>> >> >>>> --------------------------------------------------------------------- >> >>>> To unsubscribe, e-mail:[email protected] >> >>>> For additional commands, e-mail:[email protected] >> >>>> >> >>>> >> >>> --------------------------------------------------------------------- >> >>> To unsubscribe, e-mail:[email protected] >> >>> For additional commands, e-mail:[email protected] >> >>> > >
