Sure, how about: "Migrating Dropbox to Airflow, and the challenges of ETL at scale"
Abstract: Dropbox has over 20k batch jobs that run on an hourly and/or daily cadence, connected together in a large web of dependencies. Migrating this ETL to Airflow has proven to be an interesting challenge. In this talk we'll discover how we went about it, what challenges we are facing, how Airflow is helping us, and opportunities we see ahead" On Fri, May 10, 2019 at 1:05 PM Feng Lu <fen...@google.com.invalid> wrote: > Hi Gabriel, > > Awesome, could you share with us the talk title and abstract? > Thanks. > > Feng > > *From: *Gabriel Silk <gs...@dropbox.com.invalid> > *Date: *Fri, May 10, 2019 at 10:15 AM > *To: * <dev@airflow.apache.org> > *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes > > Hey Feng, > > > > I'm the tech lead on Data Infrastructure at Dropbox, and I'd like to talk > > about the challenges and opportunities in migrating large-scale ETL to > > Airflow from a system that was built in-house. > > > > > > On Fri, May 10, 2019 at 8:23 AM Szymon Przedwojski < > > szymon.przedwoj...@polidea.com> wrote: > > > > > Hi Feng, > > > > > > I'd like to submit my talk proposal for the Bay Area meetup. > > > > > > *Working title:* Migrating Apache Oozie Workflows to Apache Airflow - > the > > > developer’s perspective > > > > > > I want to talk about the Oozie-to-Airflow [1] project created by Google > > > that we at Polidea help develop. > > > It's a conversion tool written in Python that generates Airflow Python > > DAGs > > > from Oozie workflow XMLs, thus enabling easy migration of Oozie > workflows > > > to Airflow. > > > I would like to present the architecture of the solution and of the > whole > > > end-to-end system tests suite, talk about the challenges we faced when > > > translating various Oozie features to Airflow and discuss the future of > > the > > > project. > > > Moreover, I will show how the Airflow community can get involved in the > > > open-source Oozie-to-Airflow project and why it would be mutually > > > beneficial. > > > > > > *About me:* > > > I’m a Software Engineer @ Polidea in Warsaw. I’ve been programming > > > professionally for 5 years now, mainly using Python, Java and JS. > > > I have public speaking experience - I gave a couple of talks at Python > > and > > > JS meetups, at a GDG Dev Fest Warsaw event and multiple internal talks > at > > > companies I’ve worked for. > > > I’ve been involved with Airflow since August last year and I’ve > > contributed > > > several operators for different GCP services. > > > > > > [1] https://github.com/GoogleCloudPlatform/cloud-composer > > > > > > *Szymon Przedwojski* > > > Polidea | Software Engineer > > > > > > M: +48 500 330 790 > > > > > > > > > On Thu, 18 Apr 2019 at 01:08, Feng Lu <fen...@google.com.invalid> > wrote: > > > > > > > We've finalized the meetup logistics (details here > > > > < > > > > > > > > > > https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/260712102/ > > > > >). > > > > Please send in your talk proposals! > > > > Thank you. > > > > > > > > Feng > > > > > > > > On Tue, Mar 12, 2019 at 1:15 PM Feng Lu <fen...@google.com> wrote: > > > > > > > > > Hi all, > > > > > > > > > > It has been a while since we had the last Bay Area meetup and we > > > (Google) > > > > > are going to host another meetup around June this year. Will share > > with > > > > > everyone the logistic details, meanwhile, please feel free to send > in > > > > your > > > > > talk proposals! > > > > > (I see a lot of very interesting AIP ideas floating around, these > are > > > > > great candidates.) > > > > > > > > > > Looking forward to seeing you all at the event ;) > > > > > > > > > > Feng > > > > > > > > > > > > > > >
