Hello everyone. I want to resurrect the thread about adding "Ecosystem" page to Apache Airflow Webste. Since we are cleaning the various sources of documentation (Mainly CWiki) I've added the issue https://github.com/apache/airflow/issues/10274 ("Add Ecosystem page to Airflow Website"). I'd love someone picks it up and implements as a PR to the website (now since deploying the website fully automated so easy to add to the website).
I invite everyone to comment on the issue when it comes to the content - I will be happy to moderate it and include some other sources/links that people find useful. In case there will be any concerns about whether we should / should not include some links we can discuss and possibly vote it here. J. On Sat, Feb 15, 2020 at 8:53 PM Jarek Potiuk <[email protected]> wrote: > and I love the Ecosystem name. > > On Sat, Feb 15, 2020 at 8:51 PM Jarek Potiuk <[email protected]> > wrote: > >> Anyone else with an opinion against ? if not I will! proceed with adding >> some of the Powered bys. >> >> Another example of great 'powered by' enabled by Airflow (amongst others) >> is for example 'dbt' - which seems to be very popular recently - Iam >> watching it closely. >> >> it would be great to pay those tools and platforms back their trust in >> Airflow :) by linking to them. >> >> J. >> >> >> On Tue, Jan 21, 2020 at 7:35 PM Aizhamal Nurmamat kyzy < >> [email protected]> wrote: >> >>> +1 on having both "Powered by" and "Ecosystem" sections. Another apache >>> project example that has these two pages is Flink: >>> --> https://flink.apache.org/poweredby.html >>> --> https://flink.apache.org/ecosystem.html >>> >>> On Fri, Jan 17, 2020 at 6:44 AM Nathan Maynes <[email protected]> >>> wrote: >>> >>> > Some thoughts on adding an Airflow Ecosystem section to the website, or >>> > something similarly named. >>> > >>> > I like the idea of highlighting how Airflow is being used and tend to >>> > appreciate a small paragraph about HOW it is used by Company XYZ rather >>> > than just a list of companies. >>> > >>> > I also think linking to companies that provide professional support is >>> > helpful. This should be done with care as other open source projects >>> tend >>> > to lean heavily towards a single company or two which can split the >>> > community. Cassandra is one example that comes to mind. This particular >>> > area may need to be handled differently than the others. >>> > >>> > Highlighting helpful tools and resources seems like a no brainer to >>> me. The >>> > way the Groovy Language (https://groovy-lang.org/learn.html) project >>> does >>> > it seems to be a good starting point. They list everything in >>> "Ecosystem" >>> > and include other projects that support or work well with Groovy. >>> Something >>> > similar would help highlight the growing ecosystem around Airflow. >>> > >>> > On Fri, Jan 17, 2020 at 9:19 AM Jarek Potiuk <[email protected] >>> > >>> > wrote: >>> > >>> > > TL;DR; I have a proposal to add "Powered By" section (or similar) to >>> the >>> > > official Airflow Website >>> > > >>> > > This is the result of a discussion we had with other PMCs regarding >>> > whether >>> > > we should have links to external services/tools etc. using Apache >>> > Airflow. >>> > > We think devlist is the right place to bring it and all the members >>> of >>> > our >>> > > community should have a chance to share their >>> > > opinions/concerns/expectations. >>> > > >>> > > I believe we should have a page where we describe some other sources >>> of >>> > > information and some other notable uses of Airflow that are outside >>> of >>> > the >>> > > "official" source for Apache Airflow. Those are outside of the >>> community >>> > > control - but nevertheless, super useful for the users. >>> > > >>> > > The "working title" for this page could be "Others about Airflow", >>> > > "Powered by Airflow" - I am sure we could come with a good name for >>> it in >>> > > the community. >>> > > >>> > > I think we could make such a page with a very clear disclaimer that >>> those >>> > > are useful sources, but outside of the community control and we >>> should be >>> > > free to remove any of such pages from the list in case we see that >>> there >>> > > are some violations with the Apache Way. >>> > > >>> > > I thought about some general areas on such page (and we might decide >>> to >>> > > include only some of those): >>> > > >>> > > 1) Other sources of information about Airflow >>> > > - here https://github.com/jghoman/awesome-apache-airflow is >>> definitely >>> > > something that should be on the list - it's volunteering effort of >>> Jakob >>> > > but highly regarded among the community, maybe other sources could be >>> > added >>> > > here. >>> > > >>> > > 2) Who's Using Airflow: >>> > > - here we could have a list of companies that are using Airflow. We >>> > already >>> > > have it in the README.md but I think it really belongs to the website >>> > > rather than the README.md (but love to hear what others think about >>> it). >>> > > >>> > > 3) Airflow-as-a-service >>> > > - here we could add Composer, Astronomer and possibly other >>> > > "airflow-as-a-service" companies >>> > > >>> > > 4) Integrating with Airflow: >>> > > - here we could add all the workflow solutions that somehow integrate >>> > with >>> > > Airflow and use it under the hood or export DAGs to Airflow : >>> > > - our "Ooozie-2-Airflow" library >>> > > - the CWL -> Airflow (https://github.com/Barski-lab/cwl-airflow) >>> > those >>> > > CWL developers who wanted to contribute the code to Airflow, >>> > > - Databand.ai -> they build specialized ML python DSL to generate >>> > > Airflow DAGs and use Airflow as execution engine >>> > > >>> > > Those are just examples, but I think they can be very useful to be >>> > > published for the users. However it should be CRYSTAL CLEAR - those >>> are >>> > not >>> > > products/companies Apache Airflow Community has an impact on and >>> control >>> > > over. >>> > > >>> > > Let me know what you think, I look forward to hear what others think >>> > about >>> > > it. >>> > > >>> > > J. >>> > > >>> > > -- >>> > > >>> > > Jarek Potiuk >>> > > Polidea <https://www.polidea.com/> | Principal Software Engineer >>> > > >>> > > M: +48 660 796 129 <+48%20660%20796%20129> <+48660796129 >>> > <+48%20660%20796%20129>> >>> > > [image: Polidea] <https://www.polidea.com/> >>> > > >>> > >>> > >>> > -- >>> > Nathan Maynes <http://bit.ly/115hXAt> >>> > @nathanmaynes >>> > >>> >> >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/> >> >> > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
