Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #37937: URL: https://github.com/apache/airflow/pull/37937#issuecomment-2008819730 One static check to fix :). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] heartbeat recovery message [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #34457: URL: https://github.com/apache/airflow/pull/34457#issuecomment-2008744388 (or so I saw from reviewing the code myself). It's still good to keep it though in case we have a case we want to override it. -- This is an automated message from the Apache Git Servi

(airflow) branch constraints-2-8 updated: Updating constraints. Github run id:8354321829

2024-03-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-8 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-8 by this push: new db4e87ae10 Updating const

Re: [PR] heartbeat recovery message [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #34457: URL: https://github.com/apache/airflow/pull/34457#issuecomment-2008743614 > @potiuk i made the changes regarding the grace_multiplier. If I remember right, the grace_multiplier was allowed to be set as a param value in config side and I don't see that part of

Re: [PR] heartbeat recovery message [airflow]

2024-03-19 Thread via GitHub
Bowrna commented on PR #34457: URL: https://github.com/apache/airflow/pull/34457#issuecomment-2008732165 @potiuk i made the changes regarding the grace_multiplier. If I remember right, the grace_multiplier was allowed to be set as a param value in config side and I don't see that part of th

(airflow) branch main updated (95ed08ef3b -> b52e12224f)

2024-03-19 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 95ed08ef3b Made filters bar collapsible and add a full screen toggle (#38296) add b52e12224f Add upper limit to

Re: [PR] Add upper limit to planned calendar events calculation [airflow]

2024-03-19 Thread via GitHub
eladkal merged PR #38310: URL: https://github.com/apache/airflow/pull/38310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

[PR] Re-enable cassandra for Python 3.12 after the driver got libev support [airflow]

2024-03-19 Thread via GitHub
potiuk opened a new pull request, #38314: URL: https://github.com/apache/airflow/pull/38314 Cassandra has been disabled for Python 3.12 because the binary driver from cassandra has been compiled without libev support and it failed to run on Python 3.12 (released in cassandra-driver 3.29.1 t

Re: [PR] Fix in decorators and simple test case for #36057 [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #38192: URL: https://github.com/apache/airflow/pull/38192#issuecomment-2008689814 Could you please add a little more description in the commit what problem it fixes - just reference to an issue is not enough - when we merge the commit, it's good to see what problem i

[PR] Sync v2-8-stable with v2-8-test to release 2.8.3 [airflow]

2024-03-19 Thread via GitHub
jedcunningham opened a new pull request, #38313: URL: https://github.com/apache/airflow/pull/38313 Time for `2.8.4rc1`! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Activate B015 to detect pointless comparison [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #38306: URL: https://github.com/apache/airflow/pull/38306#issuecomment-2008687744 Yeah. Most of them comparing tuples to lists :) but a few require closer look. I like some of those ruff checks :) -- This is an automated message from the Apache Git Service. T

(airflow) 02/02: Update release notes for 2.8.4

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 34ea330b786457ab2ec591395a834e564dc999ec Author: Jed Cunningham AuthorDate: Tue Mar 19 23:33:11 2024 -0600

(airflow) 01/02: Update version to 2.8.4

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 16644e665ba7930307328a32a8b4679a6e2e844b Author: Jed Cunningham AuthorDate: Tue Mar 19 22:33:28 2024 -0600

(airflow) branch v2-8-test updated (10066d94d6 -> 34ea330b78)

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git from 10066d94d6 Lift the limit on importlib_resources library (#38294) new 16644e665b Update version to 2.8

Re: [PR] Adding max consecutive failed dag runs info in UI [airflow]

2024-03-19 Thread via GitHub
pateash commented on PR #38229: URL: https://github.com/apache/airflow/pull/38229#issuecomment-2008675563 > > its already present in Dag details. IMO this is very important information as it affect the dag runs( auto pauses the dag ), and should be available upfront. happy to know your thou

Re: [PR] New generaic SQL transfer operator proposal [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #38281: URL: https://github.com/apache/airflow/pull/38281#issuecomment-2008674800 Explanation of the `mypy` problem: The MyPy Issue is because for DBApiHook we have a .pyi interface as well defined (to flag any case where we change the interface. See `common.sq

Re: [PR] Add a possibility to remove the dependency from cache in a PR [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #38312: URL: https://github.com/apache/airflow/pull/38312#issuecomment-2008665816 Cache invalidation is the most difficult thing ... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Add a possibility to remove the dependency from cache in a PR [airflow]

2024-03-19 Thread via GitHub
potiuk opened a new pull request, #38312: URL: https://github.com/apache/airflow/pull/38312 When you rmeove a dependency from pyproject.toml, until your PR gets merged, it is still present in the main branch of Airflow and this is where we do a "warm" cache install of dependencies when we b

Re: [PR] Remove unused loop variable from airflow package [airflow]

2024-03-19 Thread via GitHub
jedcunningham commented on code in PR #38308: URL: https://github.com/apache/airflow/pull/38308#discussion_r1531501125 ## airflow/utils/db.py: ## @@ -90,7 +90,7 @@ "2.7.0": "405de8318b3a", "2.8.0": "10b52ebd31f7", "2.8.1": "88344c1d9134", -"2.9.0": "1fd5653699

(airflow) branch constraints-main updated: Updating constraints. Github run id:8353470113

2024-03-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 6257485763 Updating con

Re: [PR] Apply D105 to the Models Module Partly [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #38277: URL: https://github.com/apache/airflow/pull/38277#issuecomment-2008632258 I started thread at the devlist. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Activate ruff B007 to detect unused variables in the loop [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #38309: URL: https://github.com/apache/airflow/pull/38309#issuecomment-2008619588 Some checks failing :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Made filters bar collapsible and add a full screen toggle [airflow]

2024-03-19 Thread via GitHub
phanikumv merged PR #38296: URL: https://github.com/apache/airflow/pull/38296 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: Made filters bar collapsible and add a full screen toggle (#38296)

2024-03-19 Thread phanikumv
This is an automated email from the ASF dual-hosted git repository. phanikumv pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 95ed08ef3b Made filters bar collapsible and add

Re: [PR] [#38284] Update Confirmation Logic for Config Changes on Sensitive Environments Like Production [airflow]

2024-03-19 Thread via GitHub
maycuatroi commented on PR #38299: URL: https://github.com/apache/airflow/pull/38299#issuecomment-2008576733 @hussein-awala I've fixed the static check issue. Thank you for your advice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Activate ruff B004 to detect the use of hasattr(x, __call__) [airflow]

2024-03-19 Thread via GitHub
hussein-awala merged PR #38307: URL: https://github.com/apache/airflow/pull/38307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated (e0feb7b80b -> 6670437ee2)

2024-03-19 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from e0feb7b80b Fix cursor unique name surpasses Postgres identifier limit in `PostgresToGCSOperator` (#38040) a

Re: [PR] Activate B015 to detect pointless comparison [airflow]

2024-03-19 Thread via GitHub
Taragolis commented on PR #38306: URL: https://github.com/apache/airflow/pull/38306#issuecomment-2008433246 > Oups, 32 ignored checks in our tests. Whoops some of them seem broken for a while or just some change in codebase not reflected or we miss some regression found -- This is

Re: [PR] Apply D105 to the Models Module Partly [airflow]

2024-03-19 Thread via GitHub
Taragolis commented on PR #38277: URL: https://github.com/apache/airflow/pull/38277#issuecomment-2008396659 > Actually - I am not sure - do we relly want to keep that rule @Taragolis -to explain all the dunder methods ? I’m not the correct person who could explain why we select this

[PR] Add upper limit to planned calendar events calculation [airflow]

2024-03-19 Thread via GitHub
bbovenzi opened a new pull request, #38310: URL: https://github.com/apache/airflow/pull/38310 If you had a Dag that ran extremely often, like every minute, it would take a long time to fetch the data for the calendar view. And seeing that many planned events isn't too useful anyways. The ea

[PR] Activate ruff B007 to detect unused variables in the loop [airflow]

2024-03-19 Thread via GitHub
hussein-awala opened a new pull request, #38309: URL: https://github.com/apache/airflow/pull/38309 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[PR] Remove unused loop variable from airflow package [airflow]

2024-03-19 Thread via GitHub
hussein-awala opened a new pull request, #38308: URL: https://github.com/apache/airflow/pull/38308 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[PR] Activate ruff B004 to detect the use of hasattr(x, __call__) [airflow]

2024-03-19 Thread via GitHub
hussein-awala opened a new pull request, #38307: URL: https://github.com/apache/airflow/pull/38307 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[PR] Activate B015 to detect pointless comparison [airflow]

2024-03-19 Thread via GitHub
hussein-awala opened a new pull request, #38306: URL: https://github.com/apache/airflow/pull/38306 Oups, 37 ignored checks in our tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[PR] Add button to manually create dataset events [airflow]

2024-03-19 Thread via GitHub
bbovenzi opened a new pull request, #38305: URL: https://github.com/apache/airflow/pull/38305 Use the new create dataset rest api endpoint to add a "Manually create dataset event" button in the UI: https://github.com/apache/airflow/assets/4600967/d867d93d-1fbe-4267-a1fd-bce41018747e";

Re: [PR] Fix cursor unique name surpasses Postgres identifier limit in `PostgresToGCSOperator` [airflow]

2024-03-19 Thread via GitHub
boring-cyborg[bot] commented on PR #38040: URL: https://github.com/apache/airflow/pull/38040#issuecomment-2008252991 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

(airflow) branch main updated (7b2c35cee3 -> e0feb7b80b)

2024-03-19 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 7b2c35cee3 Enable `ASYNC` ruff rules (#38300) add e0feb7b80b Fix cursor unique name surpasses Postgres identifi

Re: [I] PostgresToGCSOperator with server_side_cursor use unique_name that breaks postgres identifier limit [airflow]

2024-03-19 Thread via GitHub
Taragolis closed issue #37654: PostgresToGCSOperator with server_side_cursor use unique_name that breaks postgres identifier limit URL: https://github.com/apache/airflow/issues/37654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Fix cursor unique name surpasses Postgres identifier limit in `PostgresToGCSOperator` [airflow]

2024-03-19 Thread via GitHub
Taragolis merged PR #38040: URL: https://github.com/apache/airflow/pull/38040 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: Enable `ASYNC` ruff rules (#38300)

2024-03-19 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7b2c35cee3 Enable `ASYNC` ruff rules (#38300)

Re: [PR] Enable `ASYNC` ruff rules [airflow]

2024-03-19 Thread via GitHub
hussein-awala merged PR #38300: URL: https://github.com/apache/airflow/pull/38300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[I] Retry if failed from queued should be separate from try_number [airflow]

2024-03-19 Thread via GitHub
collinmcnulty opened a new issue, #38304: URL: https://github.com/apache/airflow/issues/38304 ### Description I think Airflow should have a configureable number of attempts for re-attempting to launch a task if it was killed for being stuck in queued for too long. Currently, such re-

(airflow) branch v2-8-test updated (2b719fe35d -> 10066d94d6)

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git from 2b719fe35d Fix docker changelogs discrepancy between 2.8.3 and 2.9.0 (#38269) new 542604a15d Minor fi

(airflow) 02/02: Lift the limit on importlib_resources library (#38294)

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 10066d94d619d87477b5c1f3dad78fbebcebfb46 Author: Jarek Potiuk AuthorDate: Tue Mar 19 21:57:06 2024 +0100

(airflow) 01/02: Minor fixup on docker-stack changelog (#38293)

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 542604a15d3a54c31a000942b06fc76d598cb7c1 Author: Jed Cunningham <66968678+jedcunning...@users.noreply.github.c

Re: [PR] Separate out additional CI image checks [airflow]

2024-03-19 Thread via GitHub
potiuk commented on PR #38225: URL: https://github.com/apache/airflow/pull/38225#issuecomment-2008137872 Seems that the "big decompose of ci.yml" had stabilized a bit - so that's the small follow up looking for review -- This is an automated message from the Apache Git Service. To respond

(airflow) branch main updated (db3181c27b -> c5425b37fa)

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from db3181c27b Encrypt all trigger attributes (#38233) add c5425b37fa Minor fixup on docker-stack changelog (#3

Re: [PR] Minor fixup on docker-stack changelog [airflow]

2024-03-19 Thread via GitHub
jedcunningham merged PR #38293: URL: https://github.com/apache/airflow/pull/38293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated (2cb4209f59 -> db3181c27b)

2024-03-19 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 2cb4209f59 Lift the limit on importlib_resources library (#38294) add db3181c27b Encrypt all trigger attribu

Re: [PR] Encrypt all trigger attributes [airflow]

2024-03-19 Thread via GitHub
hussein-awala merged PR #38233: URL: https://github.com/apache/airflow/pull/38233 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [I] Airflow scheduler with statsd enabled crashes when dag_id contains unexpected characters [airflow]

2024-03-19 Thread via GitHub
cbuffett commented on issue #18010: URL: https://github.com/apache/airflow/issues/18010#issuecomment-2008133854 I'm seeing a very similar error when the stat name is too long (more than 250 characters) in Airflow 2.7.2 (masked due to sensitive data) ``` [2024-03-08, 13:49:12 UTC] {

Re: [PR] New generaic SQL transfer operator proposal [airflow]

2024-03-19 Thread via GitHub
frodo2000 commented on PR #38281: URL: https://github.com/apache/airflow/pull/38281#issuecomment-2008122313 @Taragolis @potiuk @eladkal I had changed code for that provider. Transfer logic is moved to: - DBApiHook - generic transfer - OracleHook - Oracle bulk insert transfer - S

(airflow) branch main updated (fce4ca9b93 -> 2cb4209f59)

2024-03-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from fce4ca9b93 Attempt to battle flaky DAGFileProcessorAgent test (#38289) add 2cb4209f59 Lift the limit on importlib_

Re: [PR] Lift the limit on importlib_resources library [airflow]

2024-03-19 Thread via GitHub
potiuk merged PR #38294: URL: https://github.com/apache/airflow/pull/38294 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] Mapping key "cluster-autoscaler.kubernetes.io/safe-to-evict" already defined [airflow]

2024-03-19 Thread via GitHub
hussein-awala commented on issue #38183: URL: https://github.com/apache/airflow/issues/38183#issuecomment-2008116759 I cannot reproduce it on the main branch, the issue was fixed by https://github.com/apache/airflow/pull/37917 just after releasing 1.13.0. @jedcunningham it's worth a p

Re: [I] Random api error when connecting to kubernetes [airflow]

2024-03-19 Thread via GitHub
paramjeet01 closed issue #38288: Random api error when connecting to kubernetes URL: https://github.com/apache/airflow/issues/38288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [I] Random api error when connecting to kubernetes [airflow]

2024-03-19 Thread via GitHub
paramjeet01 commented on issue #38288: URL: https://github.com/apache/airflow/issues/38288#issuecomment-2008111346 I’m sorry , this is a bug in our code and we haven’t added await to wait for xcom containers to come up. Closing this issue, thanks fir promt reply. -- This is an automated m

Re: [I] Random api error when connecting to kubernetes [airflow]

2024-03-19 Thread via GitHub
hussein-awala commented on issue #38288: URL: https://github.com/apache/airflow/issues/38288#issuecomment-2008084793 For the second log, it seems like Airflow tries to open a websocket connection to exec in the pod and read the XCom, but it doesn't find the pod for some reason (it is most l

[PR] Implement get_log_template on DagRunPydantic [airflow]

2024-03-19 Thread via GitHub
dstandish opened a new pull request, #38303: URL: https://github.com/apache/airflow/pull/38303 Also one drive-by fix removing unnecessary provide_session. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[PR] Implement clear_xcom_data on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
dstandish opened a new pull request, #38302: URL: https://github.com/apache/airflow/pull/38302 Add method needed for AIP-44 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

(airflow) branch main updated: Attempt to battle flaky DAGFileProcessorAgent test (#38289)

2024-03-19 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new fce4ca9b93 Attempt to battle flaky DAGFileProcessor

Re: [PR] Attempt to battle flaky DAGFileProcessorAgent test [airflow]

2024-03-19 Thread via GitHub
potiuk merged PR #38289: URL: https://github.com/apache/airflow/pull/38289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Implement set_state on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
dstandish commented on code in PR #38297: URL: https://github.com/apache/airflow/pull/38297#discussion_r1531035446 ## airflow/models/taskinstance.py: ## @@ -1839,6 +1839,36 @@ def key(self) -> TaskInstanceKey: """Returns a tuple that identifies the task instance uniquel

(airflow) branch constraints-2-8 updated: Updating constraints. Github run id:8348317179

2024-03-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-8 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-8 by this push: new 919c8cff96 Updating const

Re: [PR] Implement set_state on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
Taragolis commented on code in PR #38297: URL: https://github.com/apache/airflow/pull/38297#discussion_r1531029060 ## airflow/models/taskinstance.py: ## @@ -1839,6 +1839,35 @@ def key(self) -> TaskInstanceKey: """Returns a tuple that identifies the task instance uniquel

Re: [PR] Implement set_state on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
vincbeck commented on code in PR #38297: URL: https://github.com/apache/airflow/pull/38297#discussion_r1531020312 ## airflow/models/taskinstance.py: ## @@ -1839,6 +1839,36 @@ def key(self) -> TaskInstanceKey: """Returns a tuple that identifies the task instance uniquely

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-03-19 Thread via GitHub
lyuyunbo commented on code in PR #38008: URL: https://github.com/apache/airflow/pull/38008#discussion_r1531011245 ## airflow/providers/docker/operators/docker.py: ## @@ -420,6 +434,8 @@ def _run_image_with_mounts(self, target_mounts, add_tmp_variable: bool) -> list[ )

[PR] Update AWS auth manager CLI command to not disable AVP schema validation [airflow]

2024-03-19 Thread via GitHub
vincbeck opened a new pull request, #38301: URL: https://github.com/apache/airflow/pull/38301 Amazon Verified Permissions schema validation should never be disabled. It could be pretty destructive for user defined policies. --- **^ Add meaningful descri

[PR] Enable `ASYNC` ruff rules [airflow]

2024-03-19 Thread via GitHub
Taragolis opened a new pull request, #38300: URL: https://github.com/apache/airflow/pull/38300 Detect simple issues into the asyncio related code --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-03-19 Thread via GitHub
lyuyunbo commented on code in PR #38008: URL: https://github.com/apache/airflow/pull/38008#discussion_r1531011245 ## airflow/providers/docker/operators/docker.py: ## @@ -420,6 +434,8 @@ def _run_image_with_mounts(self, target_mounts, add_tmp_variable: bool) -> list[ )

Re: [PR] Implement set_state on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
dstandish commented on code in PR #38297: URL: https://github.com/apache/airflow/pull/38297#discussion_r1531000894 ## airflow/models/taskinstance.py: ## @@ -1839,6 +1839,36 @@ def key(self) -> TaskInstanceKey: """Returns a tuple that identifies the task instance uniquel

Re: [PR] Implement set_state on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
dstandish commented on code in PR #38297: URL: https://github.com/apache/airflow/pull/38297#discussion_r1530998648 ## airflow/models/taskinstance.py: ## @@ -1839,6 +1839,36 @@ def key(self) -> TaskInstanceKey: """Returns a tuple that identifies the task instance uniquel

(airflow) branch main updated (c6bc052980 -> 4506e7afe5)

2024-03-19 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from c6bc052980 Add default for `task` on TaskInstance / fix attrs on TaskInstancePydantic (#37854) add 4506e7afe5

Re: [PR] Avoid to use subprocess in asyncio loop [airflow]

2024-03-19 Thread via GitHub
Taragolis merged PR #38292: URL: https://github.com/apache/airflow/pull/38292 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Implement set_state on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
dstandish commented on code in PR #38297: URL: https://github.com/apache/airflow/pull/38297#discussion_r1530996794 ## airflow/models/taskinstance.py: ## @@ -1839,6 +1839,36 @@ def key(self) -> TaskInstanceKey: """Returns a tuple that identifies the task instance uniquel

Re: [I] AWS Batch job stuck in runnable state with no retry [airflow]

2024-03-19 Thread via GitHub
Taragolis commented on issue #38298: URL: https://github.com/apache/airflow/issues/38298#issuecomment-2007990805 I guess it something related to the [Jobs stuck in a `RUNNABLE` status ](https://docs.aws.amazon.com/batch/latest/userguide/troubleshooting.html#job_stuck_in_runnable)? -- This

Re: [PR] [#38284] Update Confirmation Logic for Config Changes on Sensitive Environments Like Production [airflow]

2024-03-19 Thread via GitHub
boring-cyborg[bot] commented on PR #38299: URL: https://github.com/apache/airflow/pull/38299#issuecomment-2007983167 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

[PR] [#38284] Update Confirmation Logic for Config Changes on Sensitive Environments Like Production [airflow]

2024-03-19 Thread via GitHub
maycuatroi opened a new pull request, #38299: URL: https://github.com/apache/airflow/pull/38299 # Description I've added a configuration option `require_confirmation_dag_change` to both the `config.yml` and the Flask App. Its default value is `False`. However, when a user sets this c

Re: [PR] Implement set_state on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
vincbeck commented on code in PR #38297: URL: https://github.com/apache/airflow/pull/38297#discussion_r1530971750 ## airflow/models/taskinstance.py: ## @@ -1839,6 +1839,36 @@ def key(self) -> TaskInstanceKey: """Returns a tuple that identifies the task instance uniquely

[I] AWS Batch job stuck in runnable state with no retry [airflow]

2024-03-19 Thread via GitHub
aiqc opened a new issue, #38298: URL: https://github.com/apache/airflow/issues/38298 ### Description I am running AWS Batch jobs using the BatchOperator. When I use Spot Instances, sometimes the job gets stuck in a "Runnable" status in the AWS Batch Queue. Probably because ther

[PR] Implement set_state on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
dstandish opened a new pull request, #38297: URL: https://github.com/apache/airflow/pull/38297 Small part towards AIP-44. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Implement get_dagrun on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
vincbeck commented on code in PR #38295: URL: https://github.com/apache/airflow/pull/38295#discussion_r1530968165 ## airflow/models/taskinstance.py: ## @@ -2108,6 +2108,14 @@ def ready_for_retry(self) -> bool: """Check on whether the task instance is in the right state

[PR] Made filters bar collapsible and add a full screen toggle [airflow]

2024-03-19 Thread via GitHub
bbovenzi opened a new pull request, #38296: URL: https://github.com/apache/airflow/pull/38296 Add a button to collapse/expand the Dag Details filter bar ![bar](https://github.com/apache/airflow/assets/4600967/e8da6a6a-1266-4e0a-98f6-d0a10dfd57f2) Combine that state with the

[PR] Implement get_dagrun on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
dstandish opened a new pull request, #38295: URL: https://github.com/apache/airflow/pull/38295 Just what the title says. Bite-size part of AIP-44. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-03-19 Thread via GitHub
potiuk commented on code in PR #38008: URL: https://github.com/apache/airflow/pull/38008#discussion_r1530952462 ## airflow/providers/docker/operators/docker.py: ## @@ -420,6 +434,8 @@ def _run_image_with_mounts(self, target_mounts, add_tmp_variable: bool) -> list[ )

Re: [PR] Avoid to use subprocess in asyncio loop [airflow]

2024-03-19 Thread via GitHub
Taragolis commented on PR #38292: URL: https://github.com/apache/airflow/pull/38292#issuecomment-2007938750 There is only two places where this ASYNC rules complains. In one test and in apache beam provider. So I just decide to fix it first into the provider, and next enable this rule, i

[PR] Lift the limit on importlib_resources library [airflow]

2024-03-19 Thread via GitHub
potiuk opened a new pull request, #38294: URL: https://github.com/apache/airflow/pull/38294 The problem with importlib_resources interacting badly with pytest_rewrite has been solved in 6.3.2 version today and we can simply skip over the bad versions. See https://github.com/python/im

Re: [PR] Avoid to use subprocess in asyncio loop [airflow]

2024-03-19 Thread via GitHub
Taragolis commented on PR #38292: URL: https://github.com/apache/airflow/pull/38292#issuecomment-2007929028 > NICE It is just found by this rules: https://docs.astral.sh/ruff/rules/#flake8-async-async -- This is an automated message from the Apache Git Service. To respond to the me

(airflow) branch main updated (d3ef6736ea -> c6bc052980)

2024-03-19 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d3ef6736ea Add Matomo as an option for analytics_tool. (#38221) add c6bc052980 Add default for `task` on TaskIn

Re: [PR] Add missing attrs / correct bad attrs on TaskInstancePydantic [airflow]

2024-03-19 Thread via GitHub
dstandish merged PR #37854: URL: https://github.com/apache/airflow/pull/37854 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[PR] Minor fixup on docker-stack changelog [airflow]

2024-03-19 Thread via GitHub
jedcunningham opened a new pull request, #38293: URL: https://github.com/apache/airflow/pull/38293 Just fixes a little whilespace, and moves 2.5.1 to be consistent with other patch release changes. Before: ![Screenshot 2024-03-19 at 12 39 08  PM](https://github.com/apache/airfl

Re: [PR] Add CLI support for bulk pause and resume of DAGs [airflow]

2024-03-19 Thread via GitHub
shahar1 commented on code in PR #38265: URL: https://github.com/apache/airflow/pull/38265#discussion_r1530904781 ## airflow/cli/cli_config.py: ## @@ -382,11 +382,19 @@ def string_lower_type(val): ), action="store_true", ) +# TODO: Deprecate in backfill command in favo

Re: [PR] Add CLI support for bulk pause and resume of DAGs [airflow]

2024-03-19 Thread via GitHub
shahar1 commented on code in PR #38265: URL: https://github.com/apache/airflow/pull/38265#discussion_r1530904781 ## airflow/cli/cli_config.py: ## @@ -382,11 +382,19 @@ def string_lower_type(val): ), action="store_true", ) +# TODO: Deprecate in backfill command in favo

Re: [PR] Add CLI support for bulk pause and resume of DAGs [airflow]

2024-03-19 Thread via GitHub
shahar1 commented on code in PR #38265: URL: https://github.com/apache/airflow/pull/38265#discussion_r1530904781 ## airflow/cli/cli_config.py: ## @@ -382,11 +382,19 @@ def string_lower_type(val): ), action="store_true", ) +# TODO: Deprecate in backfill command in favo

Re: [PR] fix(google,log): Avoid log name overriding [airflow]

2024-03-19 Thread via GitHub
shahar1 commented on PR #38071: URL: https://github.com/apache/airflow/pull/38071#issuecomment-2007873569 Please notice that static tests fail -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] EksPodOperator doesn't work with deferrable=True [airflow]

2024-03-19 Thread via GitHub
Taragolis commented on issue #38280: URL: https://github.com/apache/airflow/issues/38280#issuecomment-2007872785 It is also covered by [Amazon Managed Workflows for Apache Airflow: Specifying newer provider packages](https://docs.aws.amazon.com/mwaa/latest/userguide/connections-packages.htm

Re: [PR] Adding max consecutive failed dag runs info in UI [airflow]

2024-03-19 Thread via GitHub
bbovenzi commented on PR #38229: URL: https://github.com/apache/airflow/pull/38229#issuecomment-2007870248 > its already present in Dag details. IMO this is very important information as it affect the dag runs( auto pauses the dag ), and should be available upfront. happy to know your thou

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-03-19 Thread via GitHub
lyuyunbo commented on code in PR #38008: URL: https://github.com/apache/airflow/pull/38008#discussion_r1530899050 ## airflow/providers/docker/operators/docker.py: ## @@ -420,6 +434,8 @@ def _run_image_with_mounts(self, target_mounts, add_tmp_variable: bool) -> list[ )

(airflow) 06/07: Add missing changelog record about MySQL support in ARM image (#38267)

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 67904a59851e0f78e9e8dfb4e2a3075692425801 Author: Andrey Anshin AuthorDate: Mon Mar 18 22:33:28 2024 +0400

(airflow) 03/07: Minor security model update (#37984)

2024-03-19 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 5995565f2a09d782ddd172754d355ea3ec044fd9 Author: Jed Cunningham <66968678+jedcunning...@users.noreply.github.c

  1   2   3   >