I also created issue for it: https://github.com/apache/airflow/issues/12282
Anyone interested in taking part - please comment there! On Wed, Nov 11, 2020 at 11:59 AM Jarek Potiuk <[email protected]> wrote: > You screamed (among many others) and I listened :). And I think the time > is now to act. > > I believe the scope of "Breeze 2" should be part of the design discussion, > where we will hear other's opinions (especially the first time or fresh > contributors). > > For now, my vision is quite a bit different than yours Ash :). But I do > not want to start a design discussion just yet, I want to make breathing > space for others to chime in. > > I would love to hear many voices and interests of people before we deep > dive into what "Breeze 2" might look like. > > What I am interested in is whether: > > a) it's the right time > b) python is the right choice > c) do I have several people who would like to join and offer both - help > in designing the vision for it, as well as their time to implement it. > > I think it is crucial that those people who will be implementing it, will > be the main people who make design decisions about it, as I would love to > have a strong group of people who would like to not only take part in > developing it but also in maintaining it in the future. > > J. > > > On Wed, Nov 11, 2020 at 11:11 AM Ash Berlin-Taylor <[email protected]> wrote: > >> Omg yes. I have been screaming out for this for months. >> >> $ find scripts -name '*.sh' | xargs egrep -v '^#' | wc -l >> 6911 >> >> That's entirely too much bash for my liking by about an order of >> magnitude ;) >> >> I would also say: make breeze do less. Right now it is three major things >> : >> >> * A local development environment >> * CI runner >> * It's recently grown the ability to run airflow for developing dags. >> >> That is too much. Yes there is overlap, but it's just too much in one >> tool, and too complex as a result. Some of this should just be replaced >> with a docker-compose file (that uses published release images, not >> floating master/nightly) and users told to run that. >> >> Make it simpler, fitting a core purpose - running CI consistently should >> be it's only goal. >> >> -ash >> >> On Nov 11 2020, at 9:58 am, Jarek Potiuk <[email protected]> >> wrote: >> >> Hello Everyone, >> >> TL; DR; I was thinking for quite a while on this and I think this is the >> right time to raise that subject. It's been asked several times, why Breeze >> is not written in something else than Bash since it is "that big" or some >> people said "monstrous" :). I think it's the right time to start a >> "rewrite" project with wide community involvement and Python seems to be >> the best choice :). >> >> >> While I was opposing this while we were focusing on Airflow 2.0, and >> there are some good reasons why initially I started Breeze in Bash, I think >> with the current state of Airflow 2.0 betas, with Airflow 2.0 fully based >> on Python 3.6 and with some "stability" and "good set of features" we have >> in Breeze and a good level of modularisation we achieved - it's the right >> time to think about a rewrite. >> >> I did not raise this subject to add a distraction on top of what is >> already a lot of work for 2.0, but I think having Breeze rewritten in >> Python could be the "one more thing" that we could do - as a community to >> make 2.0 experience even better, and one that can make the community even >> closer. >> >> I was thinking that Breeze is perfect to be split into separate smaller >> pieces, describe some assumptions that we will have for its use, and turn >> it into a true community effort where a lot of people will contribute and >> where we will be able to simplify some of the stuff, and - most importantly >> - make more people from the community know about how our CI and development >> environment works and be able to solve any problems there. >> >> Breeze (and underlying bash libraries) are crucial, to get our CI working >> and I am mostly the single point of contact (and failure!) when it comes to >> that - I would love to not be one :) and I think with most of the core >> committers busy with 2.0, this is also an opportunity for more of the >> contributors to take their part in it (and eventually earn their rank to >> become committers!). For the core committers, this is an extra opportunity >> to learn how the system works, influence its design, and possibly simplify >> some parts of it - even if they will be mostly focused on 2.0. >> >> I would like to do it well - write some assumptions in a design doc, plan >> the work and split it into separate issues, and lead the effort - but I >> would love if most of the work is done by others, who would then become >> familiar with the whole of it. >> >> WDYT? Do you think it is a good idea? Do you thin k it is the right time? >> Are there some people in the community who would like to take part in it? >> >> J. >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/> >> >> > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
