Re: SQLOperator?

Thanks Alex I did notice MySQL one We are working with Oracle I guess I will have to call OracleHook from PythonOperator? On Fri, Mar 17, 2017 at 9:25 PM Alex Guziel wrote: > I'm not sure if that one went away but there are different SQL operators, > like MySqlOperator, MsSqlOperator, etc.

Re: SQLOperator?

I'm not sure if that one went away but there are different SQL operators, like MySqlOperator, MsSqlOperator, etc. that I see. Best, Alex On Fri, Mar 17, 2017 at 7:56 PM, Ruslan Dautkhanov wrote: > I can't find references to SQLOperator neither in the source code nor in > the API Reference. > >

SQLOperator?

I can't find references to SQLOperator neither in the source code nor in the API Reference. Although it is mentioned in Concepts page : https://github.com/apache/incubator-airflow/blob/master/docs/concepts.rst#operators - SqlOperator - executes a SQL command Sorry for basic questions - jus

Re: [RESULT][VOTE]Release Airflow 1.8.0 based on Airflow 1.8.0rc5

That's reasonable (treating it a bug instead of a change in behavior). Full speed ahead! On Thu, Mar 16, 2017 at 9:01 AM, Bolke de Bruin wrote: > Hello, > > Apache Airflow (incubating) 1.8.0 (RC5) has been accepted. > > 9 “+1” votes received: > > - Maxime Beauchemin (binding) > - Chris Riccomini

Re: Reminder : LatestOnlyOperator

this is nice indeed along with the new catchup option https://airflow.incubator.apache.org/scheduler.html#backfill-and-catchup Thanks Sid and Ben for adding these new options! for a complete picture, it would be nice to force only one dag run at the time. On Fri, Mar 17, 2017 at 7:33 PM, siddhar

Re: Make Scheduler More Centralized

I'm not convinced that this would add *that* much more load, we could probably change this functionality now if we wanted to. Just my two cents. On Thu, Mar 16, 2017 at 4:06 PM, Rui Wang wrote: > Thanks all your comments! > > Then looks like we should focus on scalability of scheduler now rather

Reminder : LatestOnlyOperator

With the Apache Airflow 1.8 release imminent, you may want to try out the *LatestOnlyOperator.* If you want your DAG to only run on the most recent scheduled slot, regardless of backlog, this operator will skip running downstream tasks for all DAG Runs prior to the current time slot. For example

Re: `airflow webserver -D` runs in foreground

Thanks Bolke. I didn't find a jira regarding this issue so filed a new one https://issues.apache.org/jira/browse/AIRFLOW-1004 -- Ruslan Dautkhanov On Fri, Mar 17, 2017 at 12:36 PM, Bolke de Bruin wrote: > This is a (known) bug, since the introduction of the rolling restarts. > > Bolke. > >

Airflow Impersonation feature using ssh ? ( Re: Airflow running different with different user id ? )

Hi, In the coming airflow 1.8, a feature of Impersonation will be added to allow a task using another user name. # Impersonation¶ Airflow has the ability to impersonate a unix

Rebrand to "Apache Airflow"

Hi, It was mentioned on the IPMC mailing list that we need to rebrand to “Apache Airflow” in our releases. We current release as “airflow-1.8.0+apache.incubating”, but this needs to become “apache-airflow-1.8.1+incubating” by the next release. This might be a bit troublesome for upgrades and I

Re: `airflow webserver -D` runs in foreground

This is a (known) bug, since the introduction of the rolling restarts. Bolke. > On 17 Mar 2017, at 09:48, Ruslan Dautkhanov wrote: > > $ pip freeze > airflow==*1.8.0rc5*+apache.incubating > > airflow webserver doesn't want to daemonize > > > $ airflow webserver --daemon > [2017-03-17 00:06:3

Re: Airflow best practices

FYI, we have some best practices in confluent and possibly in other places as well. I'd recommend adding to that rather than relying on email. Email can be used to mail the link :-) -s On Fri, Mar 17, 2017 at 9:33 AM, Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > Forwarding an email th

`airflow webserver -D` runs in foreground

$ pip freeze airflow==*1.8.0rc5*+apache.incubating airflow webserver doesn't want to daemonize $ airflow webserver --daemon [2017-03-17 00:06:37,553] {__init__.py:57} INFO - Using executor LocalExecutor .. skip .. Running the Gunicorn Server with: Workers: 4 sync Host: 0.0.0.0:18111 Timeout: 12

Fwd: Airflow best practices

Forwarding an email that should have been on this mailing list: -- Forwarded message -- From: Maxime Beauchemin Date: Fri, Mar 17, 2017 at 8:53 AM Subject: Re: Airflow best practices To: Shreyas Joshi Hi Shreyas, Simple Airflow scripts are simply "configuration as code" and pr

AssertionError in setup.py on git version check

Hi guys, The git version check in setup.py is failing for me on the assert on line 86 (https://github.com/apache/incubator-airflow/blob/master/setup.py#L86). It's probably me, but just wanted to check if anyone encounters this as well. These are the steps to reproduce: - pip install gitpython - c