Re: What's coming in Airflow 2.0 - NYC online meetup next week

Like for everyone else at the COVID times, fingers get fatter day-by-day ;) On Thu, May 7, 2020 at 12:17 AM Jarek Potiuk wrote: > Ah yeah... 13th of May. I wrote it with fat fingers on a small keyboard :) > > thanks :) > > J. > > > On Wed, May 6, 2020 at 11:03 PM Shaw, Damian P. < > damian.sha..

Re: What's coming in Airflow 2.0 - NYC online meetup next week

Ah yeah... 13th of May. I wrote it with fat fingers on a small keyboard :) thanks :) J. On Wed, May 6, 2020 at 11:03 PM Shaw, Damian P. < damian.sha...@credit-suisse.com> wrote: > I think there's a small typo, it should be Wednesday 13th May? > > Damian > > -Original Message- > From: J

RE: What's coming in Airflow 2.0 - NYC online meetup next week

I think there's a small typo, it should be Wednesday 13th May? Damian -Original Message- From: Jarek Potiuk Sent: Wednesday, May 6, 2020 16:22 To: dev@airflow.apache.org Subject: What's coming in Airflow 2.0 - NYC online meetup next week Next week on Wednesday 23th of May, Noon EST, si

What's coming in Airflow 2.0 - NYC online meetup next week

Next week on Wednesday 23th of May, Noon EST, six of the core committers of Apache Airflow will be talking about "What's coming in Apache Airflow 2.0". This is going to be a meeting hosted by NYC Airflow meetup, but it will be an online virtual event (obviously) https://www.meetup.com/NYC-Apache-A

Re: Setting to add choice of schedule at end or schedule at start of interval

Inspired by James, I tried this out... For others interested, here is sample dag to test it out: class MyDAG(DAG): def following_schedule(self, dttm): pen_dt = pendulum.instance(dttm).replace(second=0, microsecond=0) minutes = pen_dt.minute minutes_mod = minutes % 10

Re: CeleryExecutor and Redis

I just discovered, that Flower renders Tasks, as soon as I’m using a `BranchPythonOperator`. So that does not appear to be a setup problem. > On 6. May 2020, at 12:11, Andreas Balke wrote: > > Hi Ash, > > that appears to be OK: > > ● airflow-scheduler.service - Airflow scheduler daemon >

Re: CeleryExecutor and Redis

Hi Ash, that appears to be OK: ● airflow-scheduler.service - Airflow scheduler daemon Loaded: loaded (/lib/systemd/system/airflow-scheduler.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-05-06 10:09:20 UTC; 3s ago Main PID: 10610 (/usr/bin/python)

Re: CeleryExecutor and Redis

Your second point there would lead me to believe that the scheduler is still actually running with the default SequentialExecutor. Which config file did you edit? What output is shown when you (re)start the scheduler? Thanks, -ash On May 6 2020, at 11:00 am, Andreas Balke wrote: > Dear Airflo

CeleryExecutor and Redis

Dear Airflow community, not sure if I’m targeting the right audience here. Just trying though :) In a basic setup, like described here: https://www.cloudwalker.io/2019/09/30/airflow-scale-out-with-redis-and-celery/