Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2023-12-14 Thread via GitHub
shubham22 commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1857074044 I'd vote for simplicity and keeping things consistent irrespective of the type of the field, which means `executor_config` should _always_ override the `run_task_kwargs` for any

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2023-12-15 Thread via GitHub
o-nikolas commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1858507786 > I'd vote for simplicity and keeping things consistent irrespective of the type of the field, which means `executor_config` should _always_ override the `run_task_kwargs` for an

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2023-12-15 Thread via GitHub
shubham22 commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1858544872 @o-nikolas - it would be 2. the entire` containerOverrides` will be replaced with what is provided in `overrides`, same with `tags`. -- This is an automated message from the A

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2023-12-15 Thread via GitHub
o-nikolas commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1858555874 > @o-nikolas - it would be 2. the entire` containerOverrides` will be replaced with what is provided in `overrides` as part of the `executor_config`, same with `tags`. Int

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2023-12-15 Thread via GitHub
shubham22 commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1858568177 > if you want to just update one field for a particular task or DAG (like in this example, add some tags or update the memory for a particularly heavy task) you may wipe out 10s

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2023-12-15 Thread via GitHub
o-nikolas commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1858634526 > > if you want to just update one field for a particular task or DAG (like in this example, add some tags or update the memory for a particularly heavy task) you may wipe out 10

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2023-12-19 Thread via GitHub
ferruzzi commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1863494280 Sorry, I've been dealing with laptop issues. I like where this is heading, in general I feel like the discussion you two had is leading the right way. I guess the next logi

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2024-01-18 Thread via GitHub
syedahsn commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1899379318 Coming a bit late to the conversation, but I just want to make sure I'm following correctly. Is it true that from @shubham22 's example, the "best" final config would look somethi

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2024-01-18 Thread via GitHub
shubham22 commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1899437696 @syedahsn - confused about which approach you're calling "best" here. The main difference between your config and my suggestion is the merge and replace of _list_ configs that yo

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2024-01-19 Thread via GitHub
syedahsn commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1901325653 The difference that I was pointing out between our suggestions was that if we don't specify a key in `executor_config` which exists in `RUN_TASK_KWARGS`, that key should not be re

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2024-02-01 Thread via GitHub
o-nikolas commented on issue #35490: URL: https://github.com/apache/airflow/issues/35490#issuecomment-1922096583 I ended up going with a very opinionated approach which very closely approximates standard python update. What @shubham22 proposed [here](https://github.com/apache/airflow/issues

Re: [I] ECS Executor - Overriding Additional ECS Task Properties [airflow]

2024-02-09 Thread via GitHub
o-nikolas closed issue #35490: ECS Executor - Overriding Additional ECS Task Properties URL: https://github.com/apache/airflow/issues/35490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif