+1 (non-binding)
can’t wait to see this elegant AIP materialize!
On Thu, Nov 18 2021 at 12:18 PM, Jarek Potiuk < ja...@potiuk.com > wrote:
>
>
>
>
> +1 (binding)
>
>
>
> On Thu, Nov 18, 2021 at 6:18 PM Tomasz Urbaszek
> wrote:
> >
>
>
>>
>>
>> +1 (binding), love the idea!
>>
>>
>
>
Sounds good Lewis -
I added Metarouter integration, which follows Segment spec, so it was easy.
I'm interested to learn more about Apache Flagon :)
-Ry
On Sun, May 30, 2021 at 12:48 PM Lewis John McGibbney
wrote:
> Hi dev@,
> Over the last ~6 months we've been engaged in a migration to Apache
That is - how MSSQL performs w/ Airflow (not in general).
And also, nice work Aneesh!
On Wed, May 26, 2021 at 9:56 AM Ry Walker wrote:
> Curious - was any performance analysis done comparing how MSSQL
> performance compares w/ MySql and Postgres?
>
> On Wed, May 26, 2021 at 9:53 A
Curious - was any performance analysis done comparing how MSSQL performance
compares w/ MySql and Postgres?
On Wed, May 26, 2021 at 9:53 AM Kaxil Naik wrote:
> Yeah Kudos to Aneesh -- Sometimes it just takes a lot of patience but this
> will surely be helpful for many users using MSSQL
>
> On We
Just spent a half hour clicking through the change log - so much goodness
in this release! Great work everyone, excited to get this out to the user
community :)
-Ry
On Fri, May 21, 2021 at 11:21 AM Vikram Koka
wrote:
> Awesome!
>
> Thank you Ash, James Timmins and everyone else who contributed
o unsubscribe from this group and stop receiving emails from it, send an
> email to airflow-summit-organizers+unsubscr...@googlegroups.com.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/airflow-summit-organizers/66887FB1-73C3-418D-8A0B-2BEEE440CB81%40pinte
+1 binding - great work on this so far!
On Fri, Mar 19, 2021 at 7:43 PM James Timmins
wrote:
> Hey team,
>
> This email calls a vote to accept AIP-39:
>
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval
>
> Discussion thread:
> https://lists.apache.org/threa
+1 (binding)
On Wed, Mar 3, 2021 at 10:36 AM Kaxil Naik wrote:
> +1 (binding)
>
> On Wed, Mar 3, 2021 at 3:33 PM Brent Bovenzi
> wrote:
>
>> +1 (non-binding)
>>
>> - Brent
>>
>> On Wed, Mar 3, 2021 at 10:31 AM Ryan Hamilton
>> wrote:
>>
>>> Team,
>>>
>>> This email calls for a vote on the proj
Congrats to all the new committers - appreciate the effort!
On Mon, Mar 1, 2021 at 9:10 AM Ryan Hamilton
wrote:
> Congrats James, Elad, and Daniel!
>
> On Mon, Mar 1, 2021 at 9:06 AM Kaxil Naik wrote:
>
>> Hello Airflow Community,
>>
>> The Project Management Committee (PMC) for Apache Airflow
Congratulations Vikram, we appreciate everything you did to help get 2.0
over the finish line :)
On Tue, Jan 5, 2021 at 9:55 AM Kaxil Naik wrote:
> Hello Airflow Community,
>
> The Project Management Committee (PMC) for Apache Airflow
> has invited *Vikram Koka* to become a committer and we are
we have a webpage on it https://www.astronomer.io/airflow and a blogpost
https://www.astronomer.io/blog/introducing-airflow-2-0
On Thu, Dec 17, 2020 at 12:54 PM Shaw, Damian P. <
damian.sha...@credit-suisse.com> wrote:
> Great news! Is there a single web page that highlights these major
> feature
+1 (non-binding)
On Wed, Dec 16, 2020 at 3:38 PM Sid Anand wrote:
> +1 (binding)
>
> Great work all!
> -s
>
> On Wed, Dec 16, 2020 at 12:25 PM Kamil Breguła
> wrote:
>
>> +1 (binding)
>>
>> On Wed, Dec 16, 2020, 21:24 Elad Kalif wrote:
>>
>>> +1 (non-binding)
>>>
>>> On Wed, Dec 16, 2020 at 9:
A big thank you to the release team that has been working so hard, the
Airflow PMC, the active committers, and the hundreds of contributors who
are making Apache Airflow 2.0 a reality. It will be a great foundation for
Airflow's continued success.
It's a strong, yet non-binding +1 from me!
-Ry
Sure, I’d be happy to review it.
On Mon, Dec 7, 2020 at 1:24 PM Sounak Pradhan wrote:
> Hey everyone,
>
> I have written a blog about KubernetesPodOperator on Airflow and thought
> it would be nice to publish it on the Airflow official blogs site, is that
> possible? Should I just go ahead and c
Ha “xcom” was autocorrected to “scone” on my phone, didn’t notice :)
On Wed, Dec 2, 2020 at 10:22 AM Ry Walker wrote:
> I’m in favor of including a few backends in core, including some that can
> handle larger data, for the sake of Airflow usability and its competitive
> positioning.
I’m in favor of including a few backends in core, including some that can
handle larger data, for the sake of Airflow usability and its competitive
positioning.
This will allow us to put scone forward as a strong feature rather than how
it has been historically portrayed as flawed/limited.
On We
i like all those ideas, Jarek
On Wed, Nov 11, 2020 at 2:39 PM Jarek Potiuk
wrote:
> I read the docs (thanks Martijn!). Looks like SO would be great if we use
> more of it.
>
> We are promoting it very lightly, though. At our
> https://airflow.apache.org/community/ page we have "Stack Overflow" l
Notes from Tuesday’s call have been posted to a Github Discussion:
https://github.com/apache/airflow/discussions/12112
- Thanks for attending Patrick Cando-Almeida and Jacob Ward :) Was nice
to have people on the call who are newer to Airflow to help highlight the
deficiencies.
- We di
__twitter.com_casassaez&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=9sHnXRUkfGz7tZzoTnwwsCnQBT7JEWS9x3yvHWFZpSo&m=2XxJ21Px6dUbjMcXagRxWE1CbiyJeVeQJ88qfDCfFNk&s=TRFzqZAGPxdwbYD4a2k_L9devsCrVoHDra_jpjBOe5c&e=>
>
> On Mon, Nov 2, 2020 at 2:06 PM Jarek Potiuk
> wrote:
>
>
Hi Luciano -
Elyra looks like an interesting project — we'd love to connect and talk
through the opportunity.
You can compare your cal to mine and grab a slot here:
https://calendly.com/ryw/60min — and I'll be sure to get a few of the
Airflow PMC members to join as well.
-Ry
Ry Walk
FYI we've also made it easy to run Airflow 2 alpha locally using the astro
CLI.
1. Install the CLI:
curl -sSL https://install.astronomer.io | sudo bash -s -- v0.21.0
2. Change Dockerfile to:
FROM astronomerio/ap-airflow:2.0.0-1.dev3-buster-onbuild
3. You can install packages in requirements.tx
I+1
It’s hard to be subscribed to both dev@ and users@ (so. much. email.), and
you’d need devs subscribed to both for users to have a good experience, to
get their questions well-answered.
I unsubscribed from users@ and now that is invisible to me - good for my
focus, bad for the users.
A majori
gt;>> > We're currently working on a first draft plan for this, and
>>>>> >> lining up resources to make this happen, including a budget for
>>>>> tools and
>>>>> >> production.
>>>>> >
>>>>
Maybe modify the template to include some copy from your email in it :)
On Sun, Sep 6, 2020 at 6:56 AM Kaxil Naik wrote:
> Hi all,
>
> I have brought this topic on multiple occasions earlier too on the mailing
> list. I sincerely request all the contributors and Committers (including
> myself) t
I've seen it in other open source communities where someone tries to
maintain a somewhat detailed summary newsletter of what's going on with the
project, but weekly is a fairly brutal cadence, and it doesn't usually last
for long. In a past life (for the Meteor.js community), my team built a
tool w
Hi Sibarata -
I discussed your use case with some of our data engineers, we might have
some recommendations on how to better execute this - let us know if you
want to chat.
-Ry
Airflow Committer + Founder/CTO of Astronomer
On Mon, Sep 7, 2020 at 4:30 AM Sibabrata Pattanaik (spattana)
wrote:
>
I agree that stalebot is a best practice and we should use it, and 30d
sounds like a good starting point.
On Thu, Sep 10, 2020 at 7:56 AM Tomasz Urbaszek
wrote:
> Hi all,
>
> Currently, we have about 582 open issues on Github. The oldest opened
> in March. Do you think we should consider using s
Why not users@ mailing list:
* I personally don't currently subscribe to users@ because of email overload
* I didn't subscribe to users@ for very long as a new users, because I felt
that I could get better support elsewhere than from others on that list.
Why not Slack:
* I do visit Slack to try
gt; A while ago I also thought about moving it somewhere else into a
> different
> > file of the repo - but the Airflow website in my opinion is the best
> place
> > for it which will also probably reach more people.
> >
> > Best,
> > Felix
> >
> &g
Yes, I'm in as well, sorry for slow response!
We can probably bring a few others from Astronomer to join in too (for
example, Pete DeJoy from our team is eager to get involved here).
-Ry
On Tue, Aug 25, 2020 at 3:53 AM Jarek Potiuk
wrote:
> Cool! QP I will let you know of the plans later this
> > > On Fri, Aug 21, 2020 at 7:17 PM Jarek Potiuk <
>
> > jarek.pot...@polidea.com
>
> > > >
>
> > > > > wrote:
>
> > > > >
>
> > > > > > It's super cool indeed! I already made my suggestions and
>
mpanies and people
>
> >
>
> > > names on the main page of an Apache project.
>
> >
>
> > >
>
> >
>
> > > Hence, +1 from my side to move the list somewhere else.
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> >
I do think it’s one of the fun and impressive things about airflow, and an
easy first commit that can lead to more.
As long as it’s near the bottom it’s not doing any harm.
-Ry
On Mon, Aug 24, 2020 at 7:55 PM Jarek Potiuk
wrote:
> I think README.md for Airflow should contain some most importa
Proposal looks good to me Gerard - thanks for putting this together!
-Ry
On Fri, Aug 21, 2020 at 1:41 PM Gerard Casas Saez
wrote:
> Hi folks,
>
> I realized that the AIP process was not super clear at times. I was lucky
> when I proposed AIP-31 that Dan and Tomek guided me through most of the
>
w more about the reason why we want to nuke
> > the experimental api code soon instead of just marking it as deprecated.
> >
> > As for getting more insights into remote mode cli usage, would it make
> > sense to make it part of the airflow user survey?
> >
> >
I would think we would deprecate the old API once we say the new API is
“ready to go” - and leave it in place a while as users transition to new
API. Why is there an urgency to remove it from codebase?
On Thu, Aug 13, 2020 at 5:46 AM Ash Berlin-Taylor wrote:
> Removing the experimental is a fund
I'd say the cwiki should provide an overview of the effort, as it does now,
and that we should keep track of the work in a Github project using github
issues. The cwiki should link to that project board as the source of truth
for project status. This will help the wiki page to be perceived as up to
Important to note that the Experimental API will be replaced by a new,
fuller API that is currently under development:
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-32%3A+Airflow+REST+API
On Wed, Jul 22, 2020 at 1:37 PM Kaxil Naik wrote:
> The experimental API will be deprecated in Air
agement Committee (PMC) for Apache Airflow
> > has invited *Leah Cole* & *Ry Walker* to become a committer and we are
> > pleased
> > to announce that they have accepted.
> >
> > Being a committer enables easier contribution to the
> > project since ther
One reason to have a monorepo is for project branding, and end user
experience. But for component development experience, it's nice to have a
small, dedicated repo.
I think the git submodule approach is technically sound, but is at odds
with making the project easy to consume/understand from the e
Might be time for another community video hang out on breeze, Jarek - i'd
like to watch you using it again.
On Fri, Apr 3, 2020 at 1:47 PM Daniel Imberman
wrote:
> @jarek when you’re running on breeze how do you actually run airflow s.t.
> I can see the UI/run DAGs? It seems like it downloads ev
We should add Slack (sign up form) to https://airflow.apache.org/community/
since it really is the easiest entry point to the community. Mailing lists do
feel old school, but it's standard operating procedure for Apache projects so
it's not horrible for new people to get some experience with the
Nice work on this everyone :)
-Ry
On Wed, Dec 18, 2019 at 12:01 PM, Tomasz Urbaszek < tomasz.urbas...@polidea.com
> wrote:
>
>
>
> Hi all,
>
>
>
>
> We’ve just published results of the users survey! You can find them on our
> awesome website: http:/ / airflow. apache. org/ blog/ airflow-
Thanks Aizhamal for sparking this initiative, it's something we have been
considering for a while. Astronomer is all in for both events. We will dedicate
significant resources towards making the event valuable to the Airflow
community. Let's get started!
-Ry
Sent via Superhuman ( https://sprh.
I love it - great work :)
Sent via Superhuman ( https://sprh.mn/?vip=r...@rywalker.com )
On Thu, Oct 03, 2019 at 11:50 AM, Kamil Breguła < kamil.breg...@polidea.com >
wrote:
>
>
>
> Hello!
>
>
>
> I and Aizhamal together with Polidea team is working on a new Apache
> Airflow website. We a
+1 from me for sure. Thanks for pushing this through Aizhamal.
GCP just named their flink operator (
https://github.com/GoogleCloudPlatform/flink-on-k8s-operator )
*flink-on-k8s-operator.*
So for name, I prefer either *k8s-operator-for-airflow* or
*airflow-on-k8s-operator*. I sort of like the
Hi Eugene -
We run an Airflow deployment that executes ~6,000 hourly tasks across 32
dynamic DAGs - happy to jump on a call some time and talk about our setup,
which now includes a step to generate one file per dynamic DAG, so that each
DAG gets its own scheduler loop. Hit me up r...@astronomer
executor deployment
> at scale?
>
> Thanks,
> Austin
>
> On Wed, Jun 26, 2019 at 2:35 AM Ry Walker wrote:
>
> > If tasks are short, the k8s startup/shutdown time can be a negative
> factor.
> >
> > The nice thing about k8s executor is that you can redeploy a
If tasks are short, the k8s startup/shutdown time can be a negative factor.
The nice thing about k8s executor is that you can redeploy airflow without the
need to drain the system to ensure work won’t be affected.
Also, going all k8s gives you autoscaling.
If you’d like to try our both, you can
+1 we use pull-reminders at Astronomer, good experience. Airflow might be a bit
rough at first with 193 pending PRs, but it will help get that # down :)
Sent via Superhuman iOS ( https://sprh.mn/?vip=r...@rywalker.com )
On Tue, Jun 18 2019 at 7:31 AM, < fo...@driesprong.frl > wrote:
>
>
>
>
We're fan of (C) because:
* Sometimes, new DAGs introduce new python or system-level dependencies. It
seems cleaner to update everything together. It's a big hammer, but we've got
graceful restart of workers built into our platform. This allows us to not have
to worry about taking the system of
51 matches
Mail list logo
