Re: Blog post: Upgrading & Scaling Airflow at Robinhood

Nice post Abhishek! Glad our discussion was helpful for you guys. To share more context with the community, Airbnb had task stuck in QUEUED state problem before too. Our issues were more on the executor side. Originally it was because message lost issue in early version celery, which Alex Guziel a

Re: Blog post: Upgrading & Scaling Airflow at Robinhood

+1 Max, thanks for sharing! On Fri, Aug 9, 2019 at 2:05 PM Jarek Potiuk wrote: > +1 > > On Fri, Aug 9, 2019 at 10:54 PM Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > >> Thanks to Abhishek Ray @ Robinhood for this great post. I felt like I had >> to share it here >> >> https://robinh

Re: Blog post: Upgrading & Scaling Airflow at Robinhood

+1 On Fri, Aug 9, 2019 at 10:54 PM Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > Thanks to Abhishek Ray @ Robinhood for this great post. I felt like I had > to share it here > > https://robinhood.engineering/upgrading-scaling-airflow-at-robinhood-5b625dfaa2ee > > Max > -- Jarek Pot

Blog post: Upgrading & Scaling Airflow at Robinhood

Thanks to Abhishek Ray @ Robinhood for this great post. I felt like I had to share it here https://robinhood.engineering/upgrading-scaling-airflow-at-robinhood-5b625dfaa2ee Max

Re: Please add Airflow logo to RedBubble + stickers/swag for ApacheCons

On 07/08/2019 20:49, Aizhamal Nurmamat kyzy wrote: > Hello everyone, > > Could you please add Apache Airflow to RedBubble? Here is our logo: > http://apache.org/logos/#airflow https://www.redbubble.com/people/comdev/works/40497530-apache-airflow?asc=u Mark

RE: Data Profiling / Graphs in RBAC UI?

Thanks Ravi! I'll definitely give this a try and get back to you. Regards, Damian -Original Message- From: Ravi Kotecha [mailto:kotecha.r...@gmail.com] Sent: Friday, August 09, 2019 11:58 AM To: dev@airflow.apache.org Subject: Re: Data Profiling / Graphs in RBAC UI? Hi Damian, You mig

Re: Data Profiling / Graphs in RBAC UI?

Hi Damian, You might be able to get some of the old functionality back if you deploy Superset alongside Airflow (pointing Superset to the Airflow db) If you go down this route then please let us know how you get on. Regards, Ravi On Fri, Aug 9, 2019 at

RE: Data Profiling / Graphs in RBAC UI?

Based on the provided information I did find this under "breaking changes": https://github.com/apache/airflow/blob/master/UPDATING.md#new-webserver-ui-with-role-based-access-control . But it doesn't link to any JIRA or AIP It's a bit of a shame as they are great features. And breaking them up th

Re: Data Profiling / Graphs in RBAC UI?

Can anyone find the link to the relevant jira or AIP? On Fri, Aug 9, 2019 at 9:59 AM Deng Xiaodong wrote: > Hi Damian, > > These feature are deprecated purposely in RBAC UI for security reasons. To > my understanding there is no plan to develop replacements for them, since > it’s a deprecation r

Re: Data Profiling / Graphs in RBAC UI?

Hi Damian, These feature are deprecated purposely in RBAC UI for security reasons. To my understanding there is no plan to develop replacements for them, since it’s a deprecation rather than a “missing feature”. XD On Fri, Aug 9, 2019 at 21:54 Shaw, Damian P. < damian.sha...@credit-suisse.com>

Re: [Discuss] AIP-23 Proposal "Migration out of Travis CI"

FYI: Interesting article about the history behind GitLabCI (featuring Kamil, my friend). https://about.gitlab.com/2019/08/08/built-in-ci-cd-version-control-secret/?fbclid=IwAR2tEfqLaDXTCd1mD6XUZMX7hGYBfZcohPtI2BP3-oK_Yk_EHIXF4zLDixk On Mon, Aug 5, 2019 at 7:14 PM Jarek Potiuk wrote: > Some updat

Data Profiling / Graphs in RBAC UI?

Hi all, There are no "Data Profiling" features in the RBAC UI, is this or replacement planned to be developed? If yes, is there any existing work done or what would need to be done to contribute to this? If no, what plans are there for users who use custom graphs from this section in the curre

Data lineage - Sharing experience

Hello Airflow community, I wrote a few month back asking for data lineage feedback using Apache Airflow. At my company, we have been using Apache Airflow for several years. In the past year we have worked on data lineage solution build on top of Apache Airflow. I wanted to share with you the re