Re: Airflow health on Kubernetes

-chart/blob/master/templates/scheduler/scheduler-deployment.yaml#L94-L112 > . > This execs into the scheduler and checks on the last heartbeat from the > database. If it's been too long, the scheduler will get rebooted. > > Hope that helps. > > On Tue, Apr 21, 2020 at 3:37 AM Sergio

Airflow health on Kubernetes

Hi folks, We currently deploy Airflow on Kubernetes (using custom image, migrating to official image is planned) and we use Local executor (changing to Kubernetes executor also in plans). We meet the following problem: For cost efficiency, our testing cluster is scaled down every night. Then every

Re: Closing JIRA Issue for Merged PRs

On a slightly irrelevant note, do we ever close tickets as non-reproducable or will-not-fix? Last time I was going through open tickets I found dozens that seemed really old, really not-gonna-happen or already fixed. What actions could we take to decrease this gap? WDYT? S. On Mon, 18 Nov 2019 at

Re: [Discuss] Airflow Summits 2020

Great initiative, I am in Berlin and happy to help. S. On Mon, 18 Nov 2019 at 20:43, Aizhamal Nurmamat kyzy wrote: > Hi Jarek, > > It will be great to have Polidea's support as well. I think we need to > submit the proposal before going on Thanksgiving break to set the > desirable dates. That's

Re: Merging PRs

In my github projects I like disabling Rebase & Merge and allow only Squash & Merge. This is also an option. On Thu, 3 Oct 2019 at 09:56, Елена Лавренюк wrote: > > FYI: I added instructions below to the Contribution guide. > > > >Среда, 2 октября 2019, 18:29 +03:00 от Kaxil Naik : > > > >Hello,

Re: Issues for Hacktoberfest 2019

Hey Tomek, In two weeks there's going to be a sprint in Pycon DE and I want to have airflow as one of the projects there. So my plan is till next weekend to gather some tickets and label them accordingly to cover 2-3 different possible contributors profile. I would be glad to combine the two tasks

Re: Airflow @ApacheCon?

I live in Berlin but won't be in Apache Con. I will be though in PyCon 9-11 of October where I will try to organize a sprint session for Airflow. S. On Sun, 8 Sep 2019 at 09:42, Jarek Potiuk wrote: > Not that I am aware of unfortunately, but in case you want to go to Europe, > I have a talk at

Re: "Breeze" dev environment

Amazing! congrats and thank you. On Tue, 27 Aug 2019 at 22:25, Jarek Potiuk wrote: > Hello everyone, > > After a long time in the making I merged "Breeze" environment which is a > swiss-knife of everything testing in Airflow. It should make it even easier > to contribute to Airflow (easier to t

PyCon Berlin sprint for Airflow

There's a PyCon and Pydata conference held in October (9-13) in Berlin. There are going to be sprints for 2 days for open source projects. I will be there and I'll be happy to gather more people and work on Airflow issues. sprint proposals page

Re: [2.0 spring cleaning] Changes in import paths

My 2 cents regarding contrib. I don't see many benefits of keeping a contrib module. As a user it is super confusing that for example we have 2 slack operators. one in main operators, another in contrib. The idea that we add experimental features is fine, but in this case it lacks of good impleme

Benchmarking Airflow

Hey folks, Do we have something like airflow benchmarks? Seems that many people seem to struggle to understand the limitations of airflow (me included). Is there some existing work on bechmarking (ie define a few common cases and measure performance while increase volume of tasks)? I know it's

Re: Proposal: Restructure slack

gt;>> benefit from/opt-into, if desired)? > > >>>>>> On Sat, May 25, 2019 at 3:16 AM Jarek Potiuk > > >>> jarek.pot...@polidea.com > > >>>>>> wrote: > > >>>>>> > > >>>

Proposal: Restructure slack

Slack is a channel that could support a lot more our community. I've noticed that as a developer I get a bit overwhelmed by the amount of messages there, every time I join. This is a "good" problem, shows engaging community. What we miss though is the opportunity to catch up quickly what's going on