I really like the Idea of Tomek.
If we ever go (which is not unlikely) - some "standard" declarative way of
describing DAGs, all my security, packaging concerns are gone - and
submitting such declarative DAG via API is quite viable. Simply submitting
a Python code this way is a no-go for me :). Su
I understand the security concerns, and generally agree, but as a regular
user I always wished we could upload DAG files via an API. It opens the
door to have an upload button, which would be nice. It would make Airflow a
lot more accessible to non-engineering types.
I love the idea of implementin
In general I second what XD said. CI/CD feels better than sending DAG files
over API and the security issues arising from accepting "any python file"
are probably quite big.
However, I think this proposal can be tightly related to "declarative
DAGs". Instead of sending a DAG file, the user would s
I know this is past the deadline, but I wanted to wait to see the conversation
play out to make a more informed call.
I'd vote a (non-binding) +1.
From: Abhishek Bhakat
Sent: Thursday, August 11, 2022 2:06 AM
To: dev@airflow.apache.org
Subject: RE: [EXTERNAL][V
Hi Mocheng,
Please allow me to share a question first: so in your proposal, the API in
your plan is still accepting an Airflow DAG as the payload (just binarized
or compressed), right?
If that's the case, I may not be fully convinced: the objectives in your
proposal is about automation & programm
+1 (non-binding) for gRPC only.
On 11-Aug-2022 at 8:42:52 AM, Ash Berlin-Taylor wrote:
> +1 (binding) now with that change. Thank you very much.
>
> I'm also okay with us to proceed with only gRPC for now -- with this
> architectural change it's much easier to replace it in future if we
> want/
+1 (binding) now with that change. Thank you very much.
I'm also okay with us to proceed with only gRPC for now -- with this
architectural change it's much easier to replace it in future if we
want/need to, and for others (such as service providers like
Google/Amazon/Astronomer) to experiment
> For the security concern, is it true that "access DAG files" means
loading DAG code? If that's correct, the proposal will not introduce it
inside the api/web server, the DAG could be serialized in API client and
DAG code/files through the API would be handled as a blob but it needs to
be persiste
I made the updates in
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-44+Airflow+Internal+API
to reflect the above. As I suspected, there were really very few changes
needed, to make it "GRPC/JSON OpenAPI" agnostic. Ir does not change the
"gist" and "purpose" of the AIP and we can easily tu
Hey Ash, Andrew,
TL;DR; I slept over it to try to understand what just happened, and I have
a proposal.
* I am happy to extend my POC with the pure JSon/Open API approach -
providing that Andrew/Ash point me to some good examples where the RPC
style API has been implemented. If you could point me
10 matches
Mail list logo
