Fwd: Cannot access https://cms.apache.org/incubator/publish

Kaxil, Can you try these steps and update the airflow wiki (committer guide) based on your findings? -s Sent from Sid's iPhone Begin forwarded message: > From: Martin Gainty > Date: May 15, 2018 at 4:21:13 AM PDT > To: "san...@apache.org" > Subject: Re: Cannot access https://cms.apache.org/

Re: add a page to Airflow wiki site

I keep forgetting the images are stripped... I'm guessing you are *fenglu-g *from the list shown in the image link. https://www.dropbox.com/s/1bvh7pd1revb3x5/Screenshot%202017-08-30%2018.26.53.png?dl=0 On Wed, Aug 30, 2017 at 6:28 PM, siddharth anand wrote: > I've granted you per

Re: add a page to Airflow wiki site

I've granted you permissions. I'm guessing you are *fenglu-g* [image: Inline image 1] On Wed, Aug 30, 2017 at 5:42 PM, Feng Lu wrote: > Hi, > > We would like to share a design proposal on the wiki page > https://cwiki.apache.org/confluence/display/AIRFLOW, unfortunately it > doesn't look like

Fwd: XCOM value within a DAG

-- Forwarded message -- From: Vadzim Nemchenko Date: Thu, Jul 20, 2017 at 11:15 AM Subject: XCOM value within a DAG To: dev-ow...@airflow.incubator.apache.org Hello guys, Your Google Group is locked for creating threads, so could you help me out please with the issue below? I h

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

FYI, can anyone pictorially describe the release process (and post it on the apache airflow wiki)? I think that would eliminate a lot of confusion in the future and avoid a rehash of this email thread on the next release. -s On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah wrote: > To add, the mai

Re: Podling Report Reminder - July 2017

I've updated the Airflow report on https://wiki.apache.org/incubator/July2017 Do let me know if you have any questions. -s On Wed, Jul 5, 2017 at 7:00 AM, wrote: > Dear podling, > > This email was sent by an automated system on behalf of the Apache > Incubator PMC. It is an initial reminder to

Re: Question about updating podling report

ed to update that at some time? -s On Tue, Jul 4, 2017 at 2:01 PM, siddharth anand wrote: > Folks! > As I joined a new role (at a new company), it means that I also started > using a new laptop. I'd like to update the podling report (due tomorrow(, > but don't remember the speci

Question about updating podling report

Folks! As I joined a new role (at a new company), it means that I also started using a new laptop. I'd like to update the podling report (due tomorrow(, but don't remember the specifics for connecting via svn to the incubator repo. Anyone have instructions? I believe I need to generate a new SSH k

Re: Podling Report Reminder - July 2017

Adding this to my to-do list. On Tue, Jun 27, 2017 at 4:54 PM, wrote: > Dear podling, > > This email was sent by an automated system on behalf of the Apache > Incubator PMC. It is an initial reminder to give you plenty of time to > prepare your quarterly board report. > > The board meeting is sc

Re: DAGs dont get refreshed ?

There is a manual DAG refresh option in the UI for DAGs the UI is already aware of -- this will reload a DAG. But that's not a complete solution to the the more general DAG refresh problem. -s On Tue, Jun 20, 2017 at 6:36 PM, siddharth anand wrote: > To clarify, at Agari, we use moni

Re: DAGs dont get refreshed ?

PM, siddharth anand wrote: > We actually do restart both Web and Schedulers. I know the scheduler does > reparse the files in the dag folder, but the current state of the web ui > does require a restart. > > -s > > On Tue, Jun 20, 2017 at 6:07 PM, Ashika Umanga Umagiliya <

Re: DAGs dont get refreshed ?

We actually do restart both Web and Schedulers. I know the scheduler does reparse the files in the dag folder, but the current state of the web ui does require a restart. -s On Tue, Jun 20, 2017 at 6:07 PM, Ashika Umanga Umagiliya < umanga@gmail.com> wrote: > Greetings, > > We are using airf

Re: Passing Variables

Ah.. I completely missed the question.. in my haste to do too many things. Assuming you have a DAG named process_my_data with 3 tasks : read__from_source_table --> transform --> write_to_new_table. This dag should have a @none schedule. You could write a script to read your list of source tables

Re: Passing Variables

We use Airflow variables heavily. from airflow.models import Variable # Load an environment variable as a string ENV = Variable.get('ENV').strip() # Load an environment variable as JSON and access a JSON field named PLATFORM PLATFORM = 'EP' SSH_KEY = Variable.get('ep_platform_ssh_keys', deser

Re: New Apache Airflow meetup : in Tokyo

so popular in Japan yet, and a part of its reason is > few information in Japanese, IMHO. > The above slides are written in Japanese and really valuable for the > (potential) Airflow users in Japan. > > Regards, > > Kengo Seki > > > 2017-05-16 3:23 GMT+09:00 sidd

Re: Concurrent schedulers

I did run into "double SLA miss alarms" firing, but that was on 1.7x. I haven't tested if that is still an issue in 1.8x. -s On Tue, May 23, 2017 at 8:46 AM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > Awesome. I wasn't aware of DagRun locking, this is even better! > > Max > > On Mo

Re: Removing members from dev list?

Great. Thx Andrew. -s On Mon, May 22, 2017 at 5:23 AM, Andrew Phillips wrote: > Hi Siddarth > > How do we (PMC) remove a email recipient from the dev list? >> > > Anyone who is a moderator of the list should be able to request removal of > a subscriber by sending an email to [1]: > > {listname}-

Removing members from dev list?

How do we (PMC) remove a email recipient from the dev list? I keep getting requests to moderate the following because "kerzh...@yahoo-inc.com" is no longer at the yahoo. "kerzh...@yahoo-inc.com is no longer with Yahoo! Inc." -s

Re: New Apache Airflow meetup : in Tokyo

issues, run unit tests > and submit PRs. > > I really appreciate all speakers and audiences, and Yahoo! Japan folks > for hosting this event. > Though there's time difference between us, I hope to see some of you > core developers next time! > > Regards, > > Kengo S

Re: Article: Why Robinhood uses Airflow

Tweeted it using the Airflow account! -s On Thu, May 11, 2017 at 5:33 PM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > https://robinhood.engineering/why-robinhood-uses-airflow-aed13a9a90c8 > > Grateful to have you on board Robinhood! >

Re: IMPORTANT: I need your pypi usernames

Chris, Sorry for the delay, my user name is r39132 for both of those! -s On Tue, May 9, 2017 at 1:25 PM, Chris Riccomini wrote: > I have added the following: > > https://pypi.python.org/pypi/apache-airflow > artwr, aoen, mistercrunch > > https://testpypi.python.org/pypi/apache-airflow > misterc

Re: [VOTE] Release Airflow 1.8.1 based on Airflow 1.8.1 RC0

https://issues.apache.org/jira/browse/AIRFLOW-1121 is merged to fix the webserver pid issue.. thx Kengo! -s On Tue, Apr 18, 2017 at 6:15 PM, Hitesh Shah wrote: > -1. > > Not sure if these have been called out earlier. > > For all the bundled files with different licenses (MIT, BSD, etc), the fu

Re: Best practices on Long running process over LB

Another approach : 1. Airflow calls webservice in a fire-and-forget fashion 2. Webservice updates a message bus/stream (e.g. SQS) with result 3. An airfllow sensor pulls updates off SQS and processes them This saves airflow from polling your webservice which would in turn poll your DB. Additionall

Re: [VOTE] Release Airflow 1.8.1 based on Airflow 1.8.1 RC0

https://issues.apache.org/jira/browse/AIRFLOW-1121 Jira filed. On Tue, Apr 18, 2017 at 1:27 PM, siddharth anand wrote: > Sure. As soon as I get out of my meetings. > > -s > > On Tue, Apr 18, 2017 at 1:01 PM Chris Riccomini > wrote: > >> @Sid, can you open JIRA(s),

Re: [VOTE] Release Airflow 1.8.1 based on Airflow 1.8.1 RC0

Sure. As soon as I get out of my meetings. -s On Tue, Apr 18, 2017 at 1:01 PM Chris Riccomini wrote: > @Sid, can you open JIRA(s), and assign them as blockers to 1.8.1? > > On Tue, Apr 18, 2017 at 12:39 PM, siddharth anand > wrote: > > > I've run into a regression wi

Re: [VOTE] Release Airflow 1.8.1 based on Airflow 1.8.1 RC0

w-webserver.pid The "upgrade" process for 1.8.1. is not simply "pip install 1.8.1. tarball". It requires a "pip uninstall" of the previous 1.8.0 version followed by an new installation of 1.8.1. This could have pulled in some new dependencies that broke how this works.

Re: [VOTE] Release Airflow 1.8.1 based on Airflow 1.8.1 RC0

Hmn.. it always worked for me for any of the releases we installed. I install `pip install ` -s On Tue, Apr 18, 2017 at 10:44 AM, Chris Riccomini wrote: > @Sid, how do you enable the versioning? I've never been able to get this to > work in my environment. It always shows "Not available", even

Re: [VOTE] Release Airflow 1.8.1 based on Airflow 1.8.1 RC0

Just installed the rc in our staging env and letting it bake. FYI, I noticed that the version is not available at the following URI : /admin/versionview/ Here's a screenshot : https://www.dropbox.com/s/shdzwadb8klqwt2/Screenshot%202017-04-17%2020.57.06.png?dl=0 I installed via pip! -s On Mon, Ap

Re: Welcome @saguziel as a committer and PMC member!

Welcome Alex! On Fri, Apr 14, 2017 at 8:23 AM, Chris Riccomini wrote: > Congrats, Alex! Welcome. :) > > On Thu, Apr 13, 2017 at 7:06 PM, Dan Davydov invalid > > wrote: > > > Alex (@saguziel - AirBnB) has been making contributions and reviews for > > quite a long time now and I'm very happy to s

New Apache Airflow meetup : in Tokyo

Live in Tokyo & want to contribute to @ApacheAirflow ? Check out our new Tokyo meetup : http://bit.ly/2o7jXWF . First meetup on May 11 : https://www.meetup.com/Tokyo-Apache-Airflow-incubating-Meetup/events/238731591/ Thanks to Kengo Seki

Re: Cleanup

Edgardo, This is a great question and something that requires functionality to address. As Airflow starts getting used for bigger workloads, we need a way to clean up defunct resources. - How do we delete a dag and its related resources? - Until the recent release, the way that I stopped

Re: PTAL: Airflow 2017 April Podling Report

Just reviewed the report. Looks great! It is ready for sign-off from our mentors. -s On Wed, Apr 5, 2017 at 3:49 PM, Arthur Wiedmer wrote: > I added the following to the report, as it seemed like it was a required > question : > > How would you assess the podling's maturity?Please feel free to a

Re: 1.8.1 release

False : https://issues.apache.org/jira/browse/AIRFLOW-1055 I have a PR that works for this, but will need to add unit tests for it as well as for AIRFLOW-1013. -s On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand wrote: > Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a

Re: 1.8.1 release

AIRFLOW-1001 > >> > > >> > It looks like AIRFLOW-1017 is done, though the JIRA is not closed. > >> > > >> > The rest remain open. I will wait on the release until the remaining > >> > blockers are finished. Dan/Daniel, can you comment on status? > >&g

Re: 1.8.1 release

Ruslan, Thanks for sharing this list. I can pick a few up. I agree we should aim to get some of them into 1.8.1. -s On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov wrote: > Some of the issues I ran into while testing 1.8rc5 : > > https://issues.apache.org/jira/browse/AIRFLOW-1015 > > https:/

Re: [RESULT][VOTE]Release Airflow 1.8.0 based on Airflow 1.8.0rc5

; wrote: > >> > >>> That's reasonable (treating it a bug instead of a change in behavior). > Full > >>> speed ahead! > >>> > >>> On Thu, Mar 16, 2017 at 9:01 AM, Bolke de Bruin > wrote: > >>> > >>>> Hello, &

Re: Reminder : LatestOnlyOperator

he.org/scheduler.html#backfill-and-catchup > > Thanks Sid and Ben for adding these new options! > > for a complete picture, it would be nice to force only one dag run at the > time. > > On Fri, Mar 17, 2017 at 7:33 PM, siddharth anand > wrote: > > > With the Apache Air

Reminder : LatestOnlyOperator

With the Apache Airflow 1.8 release imminent, you may want to try out the *LatestOnlyOperator.* If you want your DAG to only run on the most recent scheduled slot, regardless of backlog, this operator will skip running downstream tasks for all DAG Runs prior to the current time slot. For example

Re: Airflow best practices

FYI, we have some best practices in confluent and possibly in other places as well. I'd recommend adding to that rather than relying on email. Email can be used to mail the link :-) -s On Fri, Mar 17, 2017 at 9:33 AM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > Forwarding an email th

Re: Airflow Committers: Landscape checks doing more harm than good?

+1 for replacing it with travis linting. On Thu, Mar 16, 2017 at 7:59 PM, Jeremiah Lowin wrote: > FWIW I recently started using yapf (https://github.com/google/yapf) with a > slightly custom config to format all of my projects. Rather than alert to > discrete linting errors and concrete style ru

Re: Adding Variables and Connections via script

Yes.. I recall that PR and I also recall adding support to manage variables via the CLI a long time back.. both of those should be in the 1.8 release. -s On Wed, Mar 8, 2017 at 10:18 AM, Nicholas Hodgkinson < nik.hodgkin...@collectivehealth.com> wrote: > Thanks everyone! This is super helpful! >

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc5

; consider the behaviour of 1.7.1 a bug. It has been present throughout all > the 1.8 rc/beta/apha series. > >> > >> So yes it is a change in behaviour whether it is a regression or an > integrity improvement is up for discussion. Either way I don’t consider it > a blo

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc5

/jira/browse/AIRFLOW-987 > >> > Hopefully fix can be part of the 1.8 release. > >> > > >> > > >> > > >> > -- > >> > Ruslan Dautkhanov > >> > > >> >> On Tue, Mar 14, 2017 at 6:19 PM, siddharth

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc5

l=0 This is a major regression because it will force everyone to use the CLI for things that they would normally use the UI for. -s -s On Tue, Mar 14, 2017 at 1:32 PM, Daniel Huang wrote: > +1 (non-binding)! > > On Tue, Mar 14, 2017 at 11:35 AM, siddharth anand > wrote: >

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc5

_SUCCESS trigger rules > > where the parent nodes where joining with a SKIP. But I can of should > have > > known this was coming. Apart of that I had a successful run last night. > > > > > > On Tue, Mar 14, 2017 at 1:37 AM siddharth anand > wrote: > > > >

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc5

I'm going to deploy this to staging now. Fab work Bolke! -s On Mon, Mar 13, 2017 at 2:16 PM, Dan Davydov wrote: > I'll test this on staging as soon as I get a chance (the testing is > non-blocking on the rc5). Bolke very much in particular :). > > On Mon, Mar 13, 2017 at 10:46 AM, Jeremiah Lowin

Re: scheduler running on multiple nodes

I did run 2 or more schedulers with Local Executors up until mid last year. There have been enough changes to the code and feature additions that I don't think this is a recommended practice at this point. Also, there is not a lot of synchronization in the scheduler to ensure this will work. -s

Re: [RESULT] [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc4

as already passed, but belated +1 (binding) > > >>> > > >>> On Tue, Feb 21, 2017 at 7:42 AM, Bolke de Bruin > > >>> wrote: > > >>> > > >>>> IPMC Voting can be found here: > > >>>> > > >>>> http://m

Re: Meetup featuring an Airflow talk tomorrow

Nice! https://twitter.com/ApacheAirflow/status/834945481440546816 Do please share slides and video so we can post both on via twitter & wiki. -s On Wed, Feb 22, 2017 at 3:00 PM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > Hi, > > Just wanted to let you know that Arthur (one of our A

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc4

+1 (binding). Thx Bolke! -s On Mon, Feb 20, 2017 at 2:51 PM, Alex Van Boxel wrote: > +1 (binding) > > On Mon, Feb 20, 2017 at 5:32 AM y...@yahoo-inc.com.INVALID > wrote: > > > > > +1 (non-binding) > > > > Thanks for all the work! > > > > YiOn Sunday, February 19, 2017, 12:52:31 PM PST, Arthur W

Re: Soliciting feedback: Using the Airflow CLI as a thin client

Hi Wilson, I'm a huge fan of the CLI and you are correct that the released current version of the CLI requires both a connection to the DB and access to the dag folder. In the new 1.8.0 release that is currently being driven by Bolke, the CLI uses the API. I'm not 100% sure that all CLIs commands

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc3

Folks! I need to change my vote.. -1 (Binding). Mark Success/Clear is broken in the UI. It's a regression. -s On Mon, Feb 13, 2017 at 10:53 AM, Alex Van Boxel wrote: > +1 (binding) > > On Mon, Feb 13, 2017 at 7:45 PM siddharth anand wrote: > > > +1 (binding) > >

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc3

+1 (binding) On Mon, Feb 13, 2017 at 8:57 AM, Chris Riccomini wrote: > +1 (binding) > > On Sun, Feb 12, 2017 at 8:54 AM, Jeremiah Lowin wrote: > > > Interesting -- I also run on Kubernetes with a git-sync sidecar, but the > > containers wait for the synced repo to apprar before starting since i

Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc3

Deployed to stage and will watch over the weekend before voting. -s On Sat, Feb 11, 2017 at 9:53 AM, Jeremiah Lowin wrote: > Boris, I submitted a PR to address your second point -- > https://github.com/apache/incubator-airflow/pull/2068. Thanks! > > On Sat, Feb 11, 2017 at 10:42 AM Boris Tyukin

Re: question regarding scheduling mechanism

Apache mail servers don't accept in-line images (a.k.a. image attachments). Please send a link to the image. -s On Tue, Feb 7, 2017 at 5:53 AM, הילה ויזן wrote: > Hi, > I've read carefully the documentation related to scheduler and scheduling > mechanism. > I found in the docs the following inf

Re: Airflow 1.8.0 Release Candidate 1

wrote: > Bolke, attached is the patch for the cgroups fix. Let me know which > branches you would like me to merge it to. If anyone has complaints about > the patch let me know (but it does not touch the core of airflow, only the > new cgroups task runner). > > On Mon

Re: Airflow Meetup @ Paypal (San Jose)

7 at 5:07 AM, Bolke de Bruin > wrote: > > > > > I might. But I would maybe be more interested in a kind of round table > / > > > panel session to discuss directions? Does that make sense? Or would you > > > like me to talk about a specific subject? >

Re: Airflow Meetup 1Q17 Talk Videos

n Mon, Feb 6, 2017 at 2:27 PM, siddharth anand wrote: > Thx George. > > <http://goog_936078273> > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Links > > I've added it to the page above and created a section for all Meetup > Videos. > > Community m

Re: Airflow 1.8.0 Release Candidate 1

| boolean| not null Indexes: "dag_stats_pkey" PRIMARY KEY, btree (dag_id, state) The PKEY is a combination of 2 provided columns, so I'm wondering why Alembic is complaining here. On Mon, Feb 6, 2017 at 4:24 PM, siddharth anand wrote: > Actually, I see the error is fur

Re: Airflow 1.8.0 Release Candidate 1

t looks like an autoincrement is missing for this table. I'm running `SQLAlchemy==1.1.4` - I see our setup.py specifies any version greater than 0.9.8 -s On Mon, Feb 6, 2017 at 4:11 PM, siddharth anand wrote: > I tried upgrading to 1.8.0rc1 from 1.7.1.3 via pip install

Re: Airflow 1.8.0 Release Candidate 1

I tried upgrading to 1.8.0rc1 from 1.7.1.3 via pip install https://dist.apache.org/repos/dist/dev/incubator/airflow/airflow-1.8.0rc1+apache.incubating.tar.gz and then running airflow upgradedb didn't quite work. First, I thought it completed successfully, then saw errors some tables were indeed mis

Re: Airflow Meetup 1Q17 Talk Videos

Thx George. https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Links I've added it to the page above and created a section for all Meetup Videos. Community members, For future reference, even if you don't plan to stream, I'd recommend recording your meetups so they can live on here. -s

Re: NYC Airflow Meetup

Great! Thanks for creating it - I've just joined so you can add me as an organizer. I've linked to it on : - https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements - https://twitter.com/ApacheAirflow/status/827743162789605382 - https://cwiki.apache.org/confluence/display/AIRFL

Re: Airflow Meetup in NYC @ Blue Apron

Hope this went well. Feel free to share videos and slides. Also, it would be great if we could create a NY Apache Airflow meetup page. Would you be interested in setting one up? It would be easier to promote a meetup page on social media than an email on this list. -s On Fri, Jan 20, 2017 at 10:3

Re: Airflow Meetup @ Paypal (San Jose)

e: > Sure, > I have created event on Meetup : > https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/ > 237412864/ > > Thanks for helping on this Siddharth. > Jayesh > > > > On Wed, Feb 1, 2017 at 7:50 PM, siddharth anand wrote: > > > I

Re: Airflow Meetup @ Paypal (San Jose)

rs for Meetup on Mar > 14" ? > > On Wed, Feb 1, 2017 at 11:59 AM Jayesh Senjaliya > wrote: > > > Yes, we are still waiting for more speakers. > > > > can anybody from Airbnb present ? > > > > anybody else ? > > > > > > - Jayesh >

Re: Airflow Meetup in NYC @ Blue Apron

Also, if you record a video, we'd be happy to place it on the wiki and promote it via our twitter feed, etc... -s On Mon, Jan 30, 2017 at 5:34 PM, Boris Tyukin wrote: > i hope you guys can share presentation slides at least for all of us who > are not in NYC > > On Mon, Jan 30, 2017 at 7:33 PM,

Re: Airflow Meetup @ Paypal (San Jose)

; wrote: > > > > > > > > > > > Boris, would you be able to attend an evening meetup on the > nights > > of > > > > > 3/15 > > > > > > or 3/16? I think attendance would be better on one of those days, > > as > > > > ma

Re: Airflow Meetup @ Paypal (San Jose)

ely, Strata has no room for us :( Paypal sounds like a great > > option. > > > > Jayesh, sounds like you're driving? :) > > > > On Tue, Jan 24, 2017 at 12:04 PM, siddharth anand > > wrote: > > > > > Russell, > > > Let us know what you l

Re: Airflow Meetup @ Paypal (San Jose)

Russell, Let us know what you learn about Strata. Even if Strata offers up rooms to communities for free (based on information such as community size, etc...), I'm doubtful they would cover food and drinks. That cost would need to be carried by a sponsor -- i.e. you'd need to find a sponsor for it

QCon London

Hi Folks! I will be attending QCon London Mar 5-8. Happy to meet locals and talk Airflow and data infrastructure if there is interest. FYI, I'm also a co-chair for QCon London and would be very interested in getting a deeper understanding of the local (London and environs) tech scene and potential

Re: Medium series: Airflow for Google Cloud

Looks like you don't have an account.. once you create one.. let me know and I will grant you admin perms on the wiki. -s On Fri, Jan 20, 2017 at 6:08 PM, siddharth anand wrote: > I've added it to https://cwiki.apache.org/confluence/display/AIRFLOW/ > Airflow+Links > > F

Re: Medium series: Airflow for Google Cloud

I've added it to https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Links Feel free to add future posts to this page. You should have access. -s On Fri, Jan 20, 2017 at 3:23 PM, Alex Van Boxel wrote: > Hey all, > > now that 1.8 is nearing release. I finally started writing about Airflo

Re: Airflow Meetup in NYC @ Blue Apron

Great to hear. Yes, please do set up an official meet-up page. You are welcome to add a few of the committer or other contributors as co-admins of the meet-up page. (e.g. https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/ ) Once the meet-up page and the event is up, we can tweet the

Re: New book covers Airflow with PySpark: Agile Data Science 2.0 (O'Reilly, 2017) AND Airflow Meetup?

> > When would be a good date, do you think? Late February? > > On Thu, Jan 19, 2017 at 5:19 PM, siddharth anand > wrote: > > > Sounds like a great idea. We are looking for someone to host the next > one.. > > once one is announced, you can sign up as a speaker.. You

Re: New book covers Airflow with PySpark: Agile Data Science 2.0 (O'Reilly, 2017) AND Airflow Meetup?

Sounds like a great idea. We are looking for someone to host the next one.. once one is announced, you can sign up as a speaker.. You are also welcome to host a meet-up if you like. -s On Thu, Jan 19, 2017 at 4:39 PM, Russell Jurney wrote: > Hello! My name is Russell Jurney. I am a relatively ne

Re: Subsequent Airflow Meetup: 2017/01/11

> > > > > > > > > > > With the new year, I was just looking to pick things back up and > > > > solicit > > > > > > presenters for the meetup. Given we're looking for two more, and > > > > > > Dan(Airbnb) and Kevi

Q1 Meetup

Dan, Kevin, Please send George your updated talk titles and abstracts. George, pls update the meetup once you have them. https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/235259523/ -s

Re: Airflow Github Mirror is not synchronizing

The repo mirror is syncing now. -s On Sat, Dec 31, 2016 at 11:22 AM siddharth anand wrote: > FYI! > I've reopened my earlier JIRA issue. It looks like multiple Apache > Projects are reporting the same. > https://issues.apache.org/jira/browse/INFRA-12949 > > Newly

Jan 2017 Airflow Podling Report Posted

Hi Folks! Here's the quarterly Podling Report for Apache Airflow. Feel free to suggest edits. If you are a committer/maintainer, you can directly edit it. https://wiki.apache.org/incubator/January2017 -s

Airflow Github Mirror is not synchronizing

FYI! I've reopened my earlier JIRA issue. It looks like multiple Apache Projects are reporting the same. https://issues.apache.org/jira/browse/INFRA-12949 Newly merged changes won't be available to contributors/users until the mirroring issue is fixed.

[AIRFLOW-676] Do not allow Pools with 0 slots

https://github.com/apache/incubator-airflow/pull/1967 Hi Folks! Would appreciate your feedback on the following pull request. If you'd like this functionality, please provide a +1 on the PR itself. -s

Re: Podling Report Reminder - January 2017

I'll put this together. -s On Thu, Dec 29, 2016 at 6:31 PM, wrote: > Dear podling, > > This email was sent by an automated system on behalf of the Apache > Incubator PMC. It is an initial reminder to give you plenty of time to > prepare your quarterly board report. > > The board meeting is sche

Re: Cold Case PR Cleanup -- Current Status

he PR load. https://cwiki.apache.org/confluence/display/AIRFLOW/Cold-Case+PR+Resolution -s On Sun, Nov 13, 2016 at 11:51 PM, siddharth anand wrote: > Current status : we have < 50 open PRs presently! > https://cwiki.apache.org/confluence/display/AIRFLOW/ > Cold-Case+PR+Resolution > >

Re: Airflow 2.0

Max, Do you have time to summarize this thread? Perhaps, publish it on the Wiki! -s On Thu, Dec 1, 2016 at 12:27 PM, Van Klaveren, Brian N. < b...@slac.stanford.edu> wrote: > With the announcement of AWS Batch (https://aws.amazon.com/batch/), and > my own selfish needs, I think it'd be really gre

Re: Merging the experimental API Framework

Bolke, Thanks for kicking this off. Is there already a design document for this? If not, can you create one? It makes sense to have a design document for this to connect multiple PRs. You can also add the information above to the same wiki -- The mailing list is not always super friendly for histo

Re: November 16 SF Bay Airflow meetup

Thx Chris. I've added it to the announcements as well. : https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements#Announcements-Nov28,2016 On Mon, Nov 28, 2016 at 11:13 AM, Chris Riccomini wrote: > Hey all, > > A recording of the meetup is now available here: > > https://wepayinc.app.b

Cold case PR cleanup deadline

Is this week, Nov 30. Friendly reminder to all maintainers.

Re: Simple Feature Request

gt; On Thu, Nov 24, 2016 at 3:51 AM, siddharth anand > wrote: > > > If we support a test query dialog, we could just execute the query via > each > > hook's pre-existing execute method. > > > > -s > > > > On Wed, Nov 23, 2016 at 2:04 PM, Bolke de

Re: Simple Feature Request

t; > Op 23 nov. 2016, om 21:28 heeft siddharth anand het > volgende geschreven: > > > > Folks! > > Here's a nice (simple) feature request if someone would like to > implement it and file a PR. > > > > On the Admin-->Connections page, you are add a conn

Simple Feature Request

Folks! Here's a nice (simple) feature request if someone would like to implement it and file a PR. On the Admin-->Connections page, you are add a connection. But how do you know if it works? How about adding a "Test Connection" button on the edit page to test if the parameters you have specified a

Re: Dynamic creation of DAG

Hi Max, Which part in the above PR is related to dynamic dags? When thinking about adding documentation about functionality, I propose the community bias towards adding working examples and test coverage. We offer a quick start (which by the way needs some updates - for example, why does it not st

Re: Airflow 2.0

have a luxury to run all the systems > on UTC. > > > On Mon, Nov 21, 2016 at 5:26 PM, siddharth anand > wrote: > > > Also, a survey will be a little less noisy and easier to summarize than > +1s > > in this email thread. > > -s (Sid) > > > > On

Re: Airflow 2.0

Also, a survey will be a little less noisy and easier to summarize than +1s in this email thread. -s (Sid) On Mon, Nov 21, 2016 at 2:25 PM, siddharth anand wrote: > Sergei, > These are some great ideas -- I would classify at least half of them as > pain points. > > Folks! > I

Re: Airflow 2.0

Sergei, These are some great ideas -- I would classify at least half of them as pain points. Folks! I suggest people (on the dev list) keep feeding this thread at least for the next 2 days. I can then float a survey based on these ideas and give the community a chance to vote so we can prioritize

Idea for a new UI feature

A sort of obvious and missing feature of Airflow (awesome UI) is knowing whether your DAGs are missing SLAs. For example, say that you have an Hourly DAG and say that you have specified the dag run timeout or sla=timedelta(hours=2), then after 2 hours, you will receive an email of the SLA miss. I

Re: Subsequent Airflow Meetup: 2017/01/11

Nov 10, 2016 at 10:17 AM, siddharth anand > wrote: > > > Kevin, > > Here's a link to the 1Q17 meet-up. > > https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/ > > 235259523/ > > > > Both upcoming meet-ups (next week at WePay and 1Q17 at C

Re: Github Mirroring currently broken

This appears to be working again! -s On Sun, Nov 20, 2016 at 12:10 PM, siddharth anand wrote: > Committers/Maintainers, > The Apache Airflow Github mirror is not synchronizing. I've filed a > ticket. It looks like, as of now, 2 other Apache projects (nifi & > brooklyn-serv

Github Mirroring currently broken

Committers/Maintainers, The Apache Airflow Github mirror is not synchronizing. I've filed a ticket. It looks like, as of now, 2 other Apache projects (nifi & brooklyn-server) have reported the same issue. https://issues.apache.org/jira/browse/INFRA-12949 This means that although we are successful

Re: Airflow 2.0

ke 1.9 a release made of smaller > packages where airflow = `(airflow-core + airflow-operators + > airflow-webserver)` or something like that. I'm thinking it would allow to > release often on airflow-operators & airflow-webserver. > > Max > > On Fri, Nov 18, 20

  1   2   3   4   >