Kamil the Jira contains many requests for new operators. To name a few:
Zeppelin Operator https://issues.apache.org/jira/browse/AIRFLOW-1032 scrapyd operator https://issues.apache.org/jira/browse/AIRFLOW-827 PostgresTableSensor https://issues.apache.org/jira/browse/AIRFLOW-1759 S3 download file operator https://issues.apache.org/jira/browse/AIRFLOW-2999 GCS rsync operator https://issues.apache.org/jira/browse/AIRFLOW-2842 There are also many requests for additional functionality and bug fixes in existed operators. Regarding the user experience there are many pending PRs in github mainly for the UI. Some are pending for weeks & months. The project doesn't have enough core committers to review them all. I noticed that many times if there are no comments to PRs the authors neglect them after a while but there isn't much we can do about it. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, February 27, 2019 7:56 PM, Kamil Breguła <kamil.breg...@polidea.com> wrote: > Very interesting results. It turns out areas that require more attention. > The most popular areas are: > > 1. Documentation - 21 comments > 2. User Experience: 11 comments > 3. More Operators: 11 comments > > Now I would like to think about what we can do about it. > > 4. The documentation has received a number of PR from my side: > My main PR is: > Automatic generation of API Reference: > > https://lists.apache.org/thread.html/9099a3b6700f04138cf607fc4c554b14f3691bae05aef52d10489b1f@<dev.airflow.apache.org> > Linking to external documentation > https://github.com/apache/airflow/pull/4655: > Fix warnings: > https://github.com/apache/airflow/pull/4585 > Validation on documentation on CI: > https://github.com/apache/airflow/pull/4593 > I wonder whether if people would base their experience on these changes, > the ratings would be better. > > I miss ideas for future improvement. Do you have any other idea for > improvement? > > 5. User Experience > User Experience is an area that has not received a lot of patches > recently. I think that this could be due to the existence of two variants > of the interface: www and www_rbac. Another reason is the lack of > automated > tests. Testing each fragment manually is very problematic. I think this is > an area that needs special attention. Now we have one variant of the > interface, so introducing changes is much simpler. > > But i have already started work and want to continue working in my spare > time. Now, I prepared a PR that extract Jinja from Javascript: > https://github.com/apache/airflow/pull/4787 As the next step, I would like > to extract the Javascipt code to separate files. Combining HTML and JS > code > into one file is not a good idea. After these steps are completed, further > improvements will be possible including: > > 1. linting for Javascript and HTML. > 2. unit tests for javascript logic, > 3. visual regression tests for generated graphs. Before or after droping > `nvd3` library > > Regarding the first improvement, It will be a small change. The > history of > commits does not exist after the www_rbac directory has become a www > directory. I think it will not be a change that will cause a lot of > discussion, unlike AIP-6. Now is the best time to force the > formatting of > the code and unify the style. > > Testing Javascript logic is problematic in applications where HTML > code is > generated by template engines. Separation of the visual layer would > significantly improve this process, but at the same time introduced > other > restrictions. Currently, I would not like to introduce it. However, > this > will be easier when the REST API is completed. > > The nvd3 library is very problematic for us due to licensing reasons. > I > think you have to think about deleting it. Visual tests allow you to > check > if graphs are always generated correctly. Making them is relatively > easy, > but there is a problem of mixing environment. The JS code is > generated by > the library in Python > > The world of Javascript has developed drastically recently and we > have to > catch up with it. > > Do you have any other visions for improvement? > > 6. More operators > Airflow is constantly evolving in this area. I do not see any significant > limitations. Today, I send a PRs that add 12 operators, 1 sensors, > However, we must observe whether there are problems that limit > development. > > Do you have ideas for new operators that will be useful? > > On Wed, Feb 27, 2019 at 5:45 PM Sid Anand r39...@gmail.com wrote: > > > > Woot! > > -s > > On Mon, Feb 25, 2019 at 9:17 PM Kevin Yang yrql...@gmail.com wrote: > > > > > +1, thank you very much. > > > On Mon, Feb 25, 2019 at 12:02 PM Maxime Beauchemin < > > > maximebeauche...@gmail.com> wrote: > > > > > > > +1, this is great and we should do it periodically! > > > > On Mon, Feb 25, 2019 at 10:42 AM Dan Davydov > > > > <ddavy...@twitter.com.invalid > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > This is very interesting and useful, big thanks for conducting the > > > > > survey! > > > > > On Mon, Feb 25, 2019 at 12:24 PM Ash Berlin-Taylor a...@apache.org > > > > > wrote: > > > > > > > > > > > Thanks for all those who answered, there's some useful answers in > > > > > > there. > > > > > > > > > > > I've done a short write up > > > > https://ash.berlintaylor.com/writings/2019/02/airflow-user-survey-2019/ > > > > > > > > Headline takeways: > > > > > > Average rating of 8.3, NPS of 35 > > > > > > Many people expecting to use @ApacheAirflow more this year > > > > > > Most people (65%) are using Celery executor, with Local and Kube > > > > > > most > > > > > > > of > > > > > > > > > > the rest > > > > > > Lots of call for sensible requests. Now we just need time to build > > > > > > them! > > > > > > > > > > > Thanks, > > > > > > -ash > > > > > > > > > > > > > On 14 Feb 2019, at 16:27, Ash Berlin-Taylor a...@apache.org > > > > > > > wrote: > > > > > > > > > > > Hi all, > > > > > > > It occurs to me I don't really know how people use Airflow, and > > > > > > > I'd > > > > > > > > like > > > > > > > > > > > to know a bit more about what sort of size clusters people run. > > > > > > > > > > > > > So if you have 5 minutes spare I'd appreciate filling out this > > > > > > > short, 7 > > > > > > > > > > > question survey: https://ashberlintaylor.typeform.com/to/hIO0Ks > > > > > > > > > > > > > I'll give it a week or so then summarise the answers back here :) > > > > > > > Thanks, > > > > > > > Ash > > -- > > Kamil Breguła > Polidea https://www.polidea.com/ | Software Engineer > > M: +48 505 458 451 <+48505458451> > E: kamil.breg...@polidea.com > [image: Polidea] https://www.polidea.com/ > > We create human & business stories through technology. > Check out our projects! https://www.polidea.com/our-work > [image: Github] https://github.com/Polidea [image: Facebook] > https://www.facebook.com/Polidea.Software [image: Twitter] > https://twitter.com/polidea [image: Linkedin] > https://www.linkedin.com/company/polidea [image: Instagram] > https://instagram.com/polidea [image: Behance] > https://www.behance.net/polidea
