One thought that occurred to me... In order to really take advantage of
async python, it may essentially require a scheduler rewrite. Certainly
there will be a lot of refactoring required. And I have not looked closely
at this. But I think chances are good that there will be a lot of things
that
_
From: Hussein Awala
Sent: Wednesday, April 10, 2024 5:44:48 PM
To: dev@airflow.apache.org
Subject: Re: [DISCUSS] Asynchronous SQLAlchemy
What you propose is exactly what `sync_to_async` does implicitly (check source
code
<https://github.com/django/asgiref/blob/db3ff43e9fa1daf7c6b1cb67db8
es where
> that helped.
>
> I imagine this approach would work in a lot of cases where only a
> synchronous driver can be used but you're working in an asynchronous
> context, hope this idea is helpful.
>
> Damian
>
>
> -Original Message-
> From: Amogh Desai
working in an asynchronous context, hope this
idea is helpful.
Damian
-Original Message-
From: Amogh Desai
Sent: Wednesday, April 10, 2024 12:33 AM
To: dev@airflow.apache.org
Subject: Re: [DISCUSS] Asynchronous SQLAlchemy
Thanks, Hussein and Ash, I am much clearer on the scope now and I am
> >
> > >
> > > On Mon, Apr 8, 2024 at 10:08 PM Daniel Standish
> > > wrote:
> > >
> > > > If nothing else, write an ugly adapter using sync_to_async?
> > > >
> > > > On Mon, Apr 8, 2024 at 1:06 PM Daniel Standish <
&g
te:
> > >
> > > > https://github.com/omnilib/aiosqlite maybe?
> > > >
> > > > On Mon, Apr 8, 2024 at 1:03 PM Scheffler Jens (XC-AS/EAE-ADA-T)
> > > > wrote:
> > > >
> > > >> I understand the „all-in“ approach as we were dropping MSSQL…
ens (XC-AS/EAE-ADA-T)
> > > > wrote:
> > > >
> > > >> I understand the „all-in“ approach as we were dropping MSSQL… how
> can
> > we
> > > >> keep one codenase bit cooe with sqlite? I assume we must support
> this
> > a
e were dropping MSSQL… how can
> we
> > >> keep one codenase bit cooe with sqlite? I assume we must support this
> at
> > >> least for dev setups.
> > >>
> > >> Sent from Outlook for iOS<https://aka.ms/o0ukef>
> > >> _
„all-in“ approach as we were dropping MSSQL… how can we
> >> keep one codenase bit cooe with sqlite? I assume we must support this at
> >> least for dev setups.
> >>
> >> Sent from Outlook for iOS<https://aka.ms/o0ukef>
> >> __________
for iOS<https://aka.ms/o0ukef>
> >> ________
> >> From: Jarek Potiuk
> >> Sent: Monday, April 8, 2024 8:30:18 PM
> >> To: dev@airflow.apache.org
> >> Subject: Re: [DISCUSS] Asynchronous SQLAlchemy
> >>
> >
ev setups.
>
> Sent from Outlook for iOS<https://aka.ms/o0ukef>
>
> From: Jarek Potiuk
> Sent: Monday, April 8, 2024 8:30:18 PM
> To: dev@airflow.apache.org
> Subject: Re: [DISCUSS] Asynchronous SQLAlchemy
>
> Yep. If we can make both Post
___
>> From: Jarek Potiuk
>> Sent: Monday, April 8, 2024 8:30:18 PM
>> To: dev@airflow.apache.org
>> Subject: Re: [DISCUSS] Asynchronous SQLAlchemy
>>
>> Yep. If we can make both Postgres and MySQL work with Async - I am also
>> all
>> for the &
pril 8, 2024 8:30:18 PM
To: dev@airflow.apache.org
Subject: Re: [DISCUSS] Asynchronous SQLAlchemy
Yep. If we can make both Postgres and MySQL work with Async - I am also all
for the "All" approach. If it means that we need to support only certain
drivers and certain versions of the DB
Yep. If we can make both Postgres and MySQL work with Async - I am also all
for the "All" approach. If it means that we need to support only certain
drivers and certain versions of the DBs - so be it. As mentioned in my
original comments (long time ago when we had MSSQL support) - this was not
real
I wholeheartedly agree with Ash that it should be all or nothing. And
*all* sounds
better to me :)
On Mon, Apr 8, 2024 at 10:54 AM Ash Berlin-Taylor wrote:
> I’m all in favour of async SQLAlchemy. We’ve built two products
> exclusively at @ Astronomer that use sqlalchemy+psycopg3+async and lo
I’m all in favour of async SQLAlchemy. We’ve built two products exclusively at
@ Astronomer that use sqlalchemy+psycopg3+async and love it. Async does take a
bit of a learning curve, but SQLA has done it nicely and it works really well.
I think this needs to be an all or nothing thing — having t
I checked the content and the PR that you attached.
The results do seem promising and I like the general idea of this approach.
But as Jarek
also mentioned on the PR:
1. Not everyone might be on the board to go all async due to certain
limitations around
access to the drivers, or corporate limita
17 matches
Mail list logo
