> However, the main issue we are trying to solve is not to provide DB creds to the customer teams, who are using Airflow as a multi-tenant orchestration platform. I've updated the doc to reflect this point as well.
Yep. We were waiting for someone like you/Uber who needs it to take the lead on that if you need it for similar purposes. Glad you are taking it on. > #2. Yeah, we would need something similar for triggerers as well, but that can be done as part of a different AIP You won't achieve your goal of "true" isolation of user code if you don't do triggerer. I think if the goal is to achieve it - it should cover both. J. On Fri, Jul 25, 2025 at 11:49 AM Sumit Maheshwari <sumeet.ma...@gmail.com> wrote: > Thanks Jarek and Ash, for the initial review. It's good to know that the > DAG processor has some preemptive measures in place to prevent access > to the DB. However, the main issue we are trying to solve is not to provide > DB creds to the customer teams, who are using Airflow as a multi-tenant > orchestration platform. I've updated the doc to reflect this point as well. > > Answering Jarek's points, > > #1. Yeah, had forgot to write about token mechanism, added that in doc, but > still how the token can be obtained (safely) is still open in my mind. I > believe the token used by task executors can be created outside of it as > well (I may be wrong here). > > #2. Yeah, we would need something similar for triggerers as well, but that > can be done as part of a different AIP > > #3. Yeah, I also believe the API should work largely. > > #4. Added that in the AIP, that instead of dag_dirs we can work with > dag_bundles and every dag-processor instance would be treated as a diff > bundle. > > Also, added points around callbacks, as these are also fetched directly > from the DB. > > On Fri, Jul 25, 2025 at 11:58 AM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > A clarification to this - the dag parser today is likely not protection > > against a dedicated malicious DAG author, but it does protect against > > casual DB access attempts - the db session is blanked out in the parsing > > process , as are the env var configs > > > > > https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/execution_time/supervisor.py#L274-L316 > > - > > is this perfect no? but it’s much more than no protection > > Oh absolutely.. This is exactly what we discussed back then in March I > > think - and the way we decided to go for 3.0 with full knowledge it's not > > protecting against all threats. > > > > On Fri, Jul 25, 2025 at 8:22 AM Ash Berlin-Taylor <a...@apache.org> > wrote: > > > > > A clarification to this - the dag parser today is likely not protection > > > against a dedicated malicious DAG author, but it does protect against > > > casual DB access attempts - the db session is blanked out in the > parsing > > > process , as are the env var configs > > > > > > https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/execution_time/supervisor.py#L274-L316 > > > - is this perfect no? but it’s much more than no protection > > > > > > > On 24 Jul 2025, at 21:56, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > > > > Currently in the DagFile processor there is no built-in protection > > > against > > > > user code from Dag Parsing to - for example - read database > > > > credentials from airflow configuration and use them to talk to DB > > > directly. > > > > > >
