Re: [PR] Simplify io.path [airflow]

2023-11-20 Thread via GitHub
bolkedebruin commented on code in PR #35747: URL: https://github.com/apache/airflow/pull/35747#discussion_r1398787330 ## airflow/io/path.py: ## @@ -123,20 +120,14 @@ def __new__(cls: type[PT], *args: str | os.PathLike, **kwargs: typing.Any) -> PT url = stringify_path

Re: [PR] Simplify io.path [airflow]

2023-11-20 Thread via GitHub
bolkedebruin commented on code in PR #35747: URL: https://github.com/apache/airflow/pull/35747#discussion_r1398791878 ## airflow/io/path.py: ## @@ -89,15 +83,18 @@ class ObjectStoragePath(CloudPath): "_hash", ) -def __new__(cls: type[PT], *args: str | os.Path

Re: [PR] Fix cache_clear behaviour in all fixtures and tests [airflow]

2023-11-20 Thread via GitHub
bolkedebruin commented on PR #35746: URL: https://github.com/apache/airflow/pull/35746#issuecomment-1818439747 Ah indeed, I was wondering how the pollution could happen at all. I wasn't aware of pytest-xdist. It feels to me like a bug in pytest-xdist, but this is a good fix. -- This is a

Re: [PR] Fix cache_clear behaviour in all fixtures and tests [airflow]

2023-11-20 Thread via GitHub
bolkedebruin merged PR #35746: URL: https://github.com/apache/airflow/pull/35746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

Re: [I] Flaky TestSerializers.test_params test [airflow]

2023-11-20 Thread via GitHub
bolkedebruin closed issue #35699: Flaky TestSerializers.test_params test URL: https://github.com/apache/airflow/issues/35699 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow) branch main updated: Fix cache_clear behaviour in all fixtures and tests (#35746)

2023-11-20 Thread bolke
This is an automated email from the ASF dual-hosted git repository. bolke pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7e7ac10947 Fix cache_clear behaviour in all fixtures

(airflow) branch constraints-main updated: Updating constraints. Github run id:6927356832

2023-11-20 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9da16cbf65 Updating con

Re: [PR] Indicate usage of the re2 regex engine in the .airflowignore documentation. [airflow]

2023-11-20 Thread via GitHub
potiuk merged PR #35663: URL: https://github.com/apache/airflow/pull/35663 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Indicate usage of the re2 regex engine in the .airflowignore documentation. [airflow]

2023-11-20 Thread via GitHub
boring-cyborg[bot] commented on PR #35663: URL: https://github.com/apache/airflow/pull/35663#issuecomment-1818587201 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

(airflow) branch main updated: Indicate usage of the re2 regex engine in the .airflowignore documentation. (#35663)

2023-11-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 861d31d897 Indicate usage of the re2 regex engine i

Re: [PR] Fix cache_clear behaviour in all fixtures and tests [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35746: URL: https://github.com/apache/airflow/pull/35746#issuecomment-1818721261 > Ah indeed, I was wondering how the pollution could happen at all. I wasn't aware of pytest-xdist. It feels to me like a bug in pytest-xdist, but this is a good fix. Just to expl

Re: [PR] Switch building airlfow packages to generic images instead of CI image [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35739: URL: https://github.com/apache/airflow/pull/35739#issuecomment-1818751144 cc: @ephraimbuddy - I would love that one to be merged before 2.8.0 cut-off as it finally closes some of the potential security loops we had in our release process and potential ways how

Re: [PR] Switch building airlfow packages to generic images instead of CI image [airflow]

2023-11-20 Thread via GitHub
ephraimbuddy commented on PR #35739: URL: https://github.com/apache/airflow/pull/35739#issuecomment-1818760129 > cc: @ephraimbuddy - I would love that one to be merged before 2.8.0 cut-off as it finally closes some of the potential security loops we had in our release process and potential

Re: [PR] Add SparkKubernetesOperator crd implementation [airflow]

2023-11-20 Thread via GitHub
hamedhsn commented on PR #22253: URL: https://github.com/apache/airflow/pull/22253#issuecomment-1818764539 > Are you on this @hamedhsn ? yeah sorry been busy last week, will sort it out this week. -- This is an automated message from the Apache Git Service. To respond to the messag

(airflow) branch switch-building-airflow-packages-to-generic-image updated (fa7926438f -> 1947165883)

2023-11-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-building-airflow-packages-to-generic-image in repository https://gitbox.apache.org/repos/asf/airflow.git omit fa7926438f Switch building airlfow packages to gneneric images instead o

(airflow) 01/01: Switch building airlfow packages to gneneric images instead of CI image

2023-11-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch switch-building-airflow-packages-to-generic-image in repository https://gitbox.apache.org/repos/asf/airflow.git commit 19471658839f678f844c4acfc46a1dd1c0a4b45b Author: Jarek Potiuk AuthorDate: S

Re: [PR] allow multiple elements in impersonation chain [airflow]

2023-11-20 Thread via GitHub
VladaZakharova commented on PR #35694: URL: https://github.com/apache/airflow/pull/35694#issuecomment-1818820713 Could you please provide your test using this operator where you are using multiple SA in `impersonation_chain` parameter as long as the results of this run? Thank you! -- Thi

Re: [PR] Switch building airlfow packages to generic images instead of CI image [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35739: URL: https://github.com/apache/airflow/pull/35739#issuecomment-1818820879 I've added the `--use-container-for-assets-compilation` flag (and added it also to the release process). -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Switch building airlfow packages to generic images instead of CI image [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35739: URL: https://github.com/apache/airflow/pull/35739#issuecomment-1818851885 Also as discussed with @ephraimbuddy on slack -> this one will make life easier for release manager that has weak/unstable/not the fastest network. It uses WAY smaller images (5x)

Re: [PR] Fix write processor_subdir in serialized_dag table [airflow]

2023-11-20 Thread via GitHub
potiuk merged PR #35661: URL: https://github.com/apache/airflow/pull/35661 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Fix write processor_subdir in serialized_dag table [airflow]

2023-11-20 Thread via GitHub
boring-cyborg[bot] commented on PR #35661: URL: https://github.com/apache/airflow/pull/35661#issuecomment-1818860975 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

(airflow) branch main updated: Fix write processor_subdir in serialized_dag table (#35661)

2023-11-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8a302d3fc1 Fix write processor_subdir in serialized

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1818945067 Shared process namespace in this case kind of violates the whole assumption the separate kerberos container here is introduced for. The whole idea is that "airflow" containers do not see

Re: [PR] Switch building airlfow packages to generic images instead of CI image [airflow]

2023-11-20 Thread via GitHub
potiuk merged PR #35739: URL: https://github.com/apache/airflow/pull/35739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Switch building airlfow packages to gneneric images instead of CI image (#35739)

2023-11-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6905c77818 Switch building airlfow packages to gnen

(airflow) branch switch-building-airflow-packages-to-generic-image deleted (was 1947165883)

2023-11-20 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch switch-building-airflow-packages-to-generic-image in repository https://gitbox.apache.org/repos/asf/airflow.git was 1947165883 Switch building airlfow packages to gneneric images i

Re: [I] Dynamic Task Mapping: transforming dict data with map results in KeyError [airflow]

2023-11-20 Thread via GitHub
nathadfield closed issue #35361: Dynamic Task Mapping: transforming dict data with map results in KeyError URL: https://github.com/apache/airflow/issues/35361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

(airflow) branch v2-8-test created (now 6fc4a9cb3c)

2023-11-20 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git at 6fc4a9cb3c Update default branches for 2-8 This branch includes the following new commits: new 6fc

(airflow) 01/01: Update default branches for 2-8

2023-11-20 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 6fc4a9cb3c8923c4455039756bfbbd636fbc7883 Author: Ephraim Anierobi AuthorDate: Mon Nov 20 13:31:03 2023 +010

(airflow) branch v2-8-stable created (now 6fc4a9cb3c)

2023-11-20 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-8-stable in repository https://gitbox.apache.org/repos/asf/airflow.git at 6fc4a9cb3c Update default branches for 2-8 No new revisions were added by this update.

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-20 Thread via GitHub
amoghrajesh commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1818979974 Thank you for the detailed review @potiuk. This is extremely helpful to me. > Another option (probably simpler to implement) is to keep some kind of "shutdown" lock file in t

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-20 Thread via GitHub
amoghrajesh commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1818986629 >we should figure out a different way how to communicate with the kerberos refreshing process and stop it - one of the options is to let the running process be stoppable by sending

[PR] Add v2-8 branches to codecov.yml and .asf.yaml [airflow]

2023-11-20 Thread via GitHub
ephraimbuddy opened a new pull request, #35750: URL: https://github.com/apache/airflow/pull/35750 Time for `2.8.0`! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

(airflow) branch constraints-2-8 created (now c7abfb4fb1)

2023-11-20 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch constraints-2-8 in repository https://gitbox.apache.org/repos/asf/airflow.git at c7abfb4fb1 Updating constraints. Github run id:4693629049 No new revisions were added by this upda

[PR] Update minor release command [airflow]

2023-11-20 Thread via GitHub
ephraimbuddy opened a new pull request, #35751: URL: https://github.com/apache/airflow/pull/35751 When creating the constraints file for the new release, pull the constraints-main first -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1819032429 I think we have ways to generate different resources for 1.28+ . I believe helm takes K8s version into account when generating resources. Look in our chart I think we have a few cases o

Re: [PR] Add XCom tab to Grid [airflow]

2023-11-20 Thread via GitHub
bbovenzi commented on code in PR #35719: URL: https://github.com/apache/airflow/pull/35719#discussion_r1399254518 ## airflow/www/static/js/dag/details/index.tsx: ## @@ -138,6 +146,7 @@ const Details = ({ const isGroupOrMappedTaskSummary = isGroup || isMappedTaskSummary; co

Re: [PR] Remove backcompat with Airflow 2.3/2.4 in providers [airflow]

2023-11-20 Thread via GitHub
Taragolis merged PR #35727: URL: https://github.com/apache/airflow/pull/35727 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated (0d1c8de78c -> d8075cd04c)

2023-11-20 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 0d1c8de78c Update minor release command (#35751) add d8075cd04c Remove backcompat with Airflow 2.3/2.4 in provi

[PR] OpenLineage integration tried to use non-existed method in SnowflakeHook [airflow]

2023-11-20 Thread via GitHub
Taragolis opened a new pull request, #35752: URL: https://github.com/apache/airflow/pull/35752 OpenLineage integration tried to use non existed hook method, which failed if correct configure inheritance for DBApiHook (upcoming PR) ```console airflow/providers/snowfl

(airflow) branch main updated: Add v2-8 branches to codecov.yml and .asf.yaml (#35750)

2023-11-20 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c07c5925e9 Add v2-8 branches to codecov.ym

Re: [PR] Add v2-8 branches to codecov.yml and .asf.yaml [airflow]

2023-11-20 Thread via GitHub
ephraimbuddy merged PR #35750: URL: https://github.com/apache/airflow/pull/35750 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

Re: [PR] Update minor release command [airflow]

2023-11-20 Thread via GitHub
ephraimbuddy merged PR #35751: URL: https://github.com/apache/airflow/pull/35751 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

(airflow) branch main updated (c07c5925e9 -> 0d1c8de78c)

2023-11-20 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from c07c5925e9 Add v2-8 branches to codecov.yml and .asf.yaml (#35750) add 0d1c8de78c Update minor release co

Re: [PR] Add XCom tab to Grid [airflow]

2023-11-20 Thread via GitHub
bbovenzi commented on code in PR #35719: URL: https://github.com/apache/airflow/pull/35719#discussion_r1399260330 ## airflow/www/static/js/dag/details/taskInstance/Xcom/XcomEntry.tsx: ## @@ -0,0 +1,82 @@ +/*! + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] Add XCom tab to Grid [airflow]

2023-11-20 Thread via GitHub
bbovenzi commented on PR #35719: URL: https://github.com/apache/airflow/pull/35719#issuecomment-1819150225 Thanks so much for doing it. I just have a few comments, but overall looks good! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Add XCom tab to Grid [airflow]

2023-11-20 Thread via GitHub
bbovenzi commented on code in PR #35719: URL: https://github.com/apache/airflow/pull/35719#discussion_r1399266549 ## airflow/www/static/js/dag/details/taskInstance/Xcom/index.tsx: ## @@ -0,0 +1,110 @@ +/*! + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] Base date for fetching dag grid view must include selected run_id [airflow]

2023-11-20 Thread via GitHub
bbovenzi commented on PR #34887: URL: https://github.com/apache/airflow/pull/34887#issuecomment-1819160336 > > I don't see why we need a custom context provider. I would prefer we try to have a `useEffect` inside of `useGridData` to detect only the first time `runId` is specified in the url

[PR] Fix DataFusion example type annotations [airflow]

2023-11-20 Thread via GitHub
Taragolis opened a new pull request, #35753: URL: https://github.com/apache/airflow/pull/35753 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideline

Re: [PR] Render DAGCode in the Grid View as a tab [airflow]

2023-11-20 Thread via GitHub
bbovenzi commented on PR #31113: URL: https://github.com/apache/airflow/pull/31113#issuecomment-1819167184 I would prefer to keep it to a page refetch since changing the DAG code because we're reparsing the dag and those changes could affect any number of ways we render the entire DAG page.

[PR] Added table of content in CONTRIBUTE.md and enhanced the documentation. [airflow-site]

2023-11-20 Thread via GitHub
KesharwaniArpita opened a new pull request, #897: URL: https://github.com/apache/airflow-site/pull/897 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Add borderWidthRight to grid for Firefox scrollbar [airflow]

2023-11-20 Thread via GitHub
vchiapaikeo commented on PR #35346: URL: https://github.com/apache/airflow/pull/35346#issuecomment-1819243364 @eladkal , wdyt? Okay to merge here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] OpenLineage integration tried to use non-existed method in SnowflakeHook [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35752: URL: https://github.com/apache/airflow/pull/35752#issuecomment-1819301766 Should we have some more unit tests for those @mobuchowski ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Added table of content in CONTRIBUTE.md and enhanced the documentation. [airflow-site]

2023-11-20 Thread via GitHub
potiuk commented on PR #897: URL: https://github.com/apache/airflow-site/pull/897#issuecomment-1819306207 This should be generated automtatically. We hve pre-commit for that in apache-airflow - so we should copy the pre-commit from there and generate the TOC automatically with it - this way

Re: [PR] OpenLineage integration tried to use non-existed method in SnowflakeHook [airflow]

2023-11-20 Thread via GitHub
Taragolis commented on PR #35752: URL: https://github.com/apache/airflow/pull/35752#issuecomment-1819325999 Yeah it would be nice to have unit tests for that. I'm not familiar with OpenLineage integration otherwise I would be write this tests and cover this part into the [hook.py](ht

Re: [PR] OpenLineage integration tried to use non-existed method in SnowflakeHook [airflow]

2023-11-20 Thread via GitHub
Taragolis merged PR #35752: URL: https://github.com/apache/airflow/pull/35752 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: OpenLineage integration tried to use non-existed method in SnowflakeHook (#35752)

2023-11-20 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f8dd192483 OpenLineage integration tried to use

[PR] Remove backcompat inheritance for DbApiHook [airflow]

2023-11-20 Thread via GitHub
Taragolis opened a new pull request, #35754: URL: https://github.com/apache/airflow/pull/35754 Follow-up: https://github.com/apache/airflow/pull/35727, https://github.com/apache/airflow/pull/35752 Remove inheritance from backcompat classes and fix static checks

[I] Allow Google Hooks to use AnonymousCredentials [airflow]

2023-11-20 Thread via GitHub
jvaesteves opened a new issue, #35755: URL: https://github.com/apache/airflow/issues/35755 ### Description Currently, all Google operations demands valid credentials to connect with GCP, but when trying to mock Google services using emulators, we are giving the opportunity to redirec

Re: [I] Users Error [airflow]

2023-11-20 Thread via GitHub
Rafnel commented on issue #34970: URL: https://github.com/apache/airflow/issues/34970#issuecomment-1819389757 Wanted to add another datapoint, we are on Airflow 2.6.3 and our users/roles page has been broken for a long time, same error as above. Downgrading to WTForms 3.0.1 fixed the issue.

(airflow) branch constraints-main updated: Updating constraints. Github run id:6932564928

2023-11-20 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 5da4d10ba6 Updating con

Re: [PR] Add XCom tab to Grid [airflow]

2023-11-20 Thread via GitHub
hduong-mwam commented on code in PR #35719: URL: https://github.com/apache/airflow/pull/35719#discussion_r1399502342 ## airflow/www/static/js/dag/details/index.tsx: ## @@ -138,6 +146,7 @@ const Details = ({ const isGroupOrMappedTaskSummary = isGroup || isMappedTaskSummary;

Re: [PR] Render DAGCode in the Grid View as a tab [airflow]

2023-11-20 Thread via GitHub
hduong-mwam commented on PR #31113: URL: https://github.com/apache/airflow/pull/31113#issuecomment-1819468029 I made this comment before I fully completed my XCom tab PR. So, I agree that probably best to keep the page refresh for DAG code as there isn't a good "event" that we can use as tr

Re: [PR] OpenLineage integration tried to use non-existed method in SnowflakeHook [airflow]

2023-11-20 Thread via GitHub
mobuchowski commented on PR #35752: URL: https://github.com/apache/airflow/pull/35752#issuecomment-1819473414 Yeah... I guess this worked because the error was caught later: https://github.com/apache/airflow/blob/b75f9e880614fa0427e7d24a1817955f5de658b3/airflow/providers/common/sql/operators

Re: [PR] OpenLineage integration tried to use non-existed method in SnowflakeHook [airflow]

2023-11-20 Thread via GitHub
JDarDagran commented on PR #35752: URL: https://github.com/apache/airflow/pull/35752#issuecomment-1819505026 Two things failed: insufficient tests and except block that should be rather changed into check with `hasattr`. I'll fix that. -- This is an automated message from the Apach

[I] Google Cloud PubSubCreateTopicOperator should support schema and message retention params [airflow]

2023-11-20 Thread via GitHub
dmedora opened a new issue, #35757: URL: https://github.com/apache/airflow/issues/35757 ### Description The PubSub [Topics Create API](https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.topics/create#request-body) supports the SchemaSettings and messageRetentionDuration.

[I] `DockerOperator`'s `env_file` param accepts "file contents as text" and not "path to dotenv file" [airflow]

2023-11-20 Thread via GitHub
Kache opened a new issue, #35758: URL: https://github.com/apache/airflow/issues/35758 ### Apache Airflow version 2.7.3 ### What happened `DockerOperator`'s [`env_file` param](https://github.com/apache/airflow/blob/f8dd19248345ac259153812c68b65fc39accfcfd/airflow/provider

[PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
potiuk opened a new pull request, #35759: URL: https://github.com/apache/airflow/pull/35759 When we run airflow package preparation in docker containers and asset compilation, we use docker containers - similarly as breeze CI image, those constainers use root user in container. This means t

Re: [I] `DockerOperator`'s `env_file` param accepts "file contents as text" and not "path to dotenv file" [airflow]

2023-11-20 Thread via GitHub
Taragolis commented on issue #35758: URL: https://github.com/apache/airflow/issues/35758#issuecomment-1819574511 As far as I remember this attribute utilise Templated Fields mechanism and expected path to the template file .. which evaluated to the string, as you mention into the example.

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35759: URL: https://github.com/apache/airflow/pull/35759#issuecomment-1819578579 World is a bit better place with every 150 or so less lines of bash script :D -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Fix service account impersonation with computeEngineSSHHook (google provider) and IAP tunnel [airflow]

2023-11-20 Thread via GitHub
ginolegigot commented on PR #35136: URL: https://github.com/apache/airflow/pull/35136#issuecomment-1819606385 Hey! I think i understood how to setup a proper airflow dev environment, launch tests and I think my unit test even though it might not optimal at least is functional (first time

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
utkarsharma2 commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399605295 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
utkarsharma2 commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399605295 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
utkarsharma2 commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399605295 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
utkarsharma2 commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399606213 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
potiuk commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399613884 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more c

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35759: URL: https://github.com/apache/airflow/pull/35759#issuecomment-1819635962 I also removed "fix ownership" step from CI as I moved the ownership fixing as "always happening" after anything is run in docker on linux - so we do nothave to run separate steps for th

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
potiuk commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399621198 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more c

Re: [PR] Remove backcompat inheritance for DbApiHook [airflow]

2023-11-20 Thread via GitHub
potiuk commented on code in PR #35754: URL: https://github.com/apache/airflow/pull/35754#discussion_r1399628334 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -120,21 +118,7 @@ def connect(self, host: str, port: int, username: str, schema: str) -> Any: """ -#

Re: [PR] Remove backcompat inheritance for DbApiHook [airflow]

2023-11-20 Thread via GitHub
potiuk commented on code in PR #35754: URL: https://github.com/apache/airflow/pull/35754#discussion_r1399630189 ## airflow/providers/trino/hooks/trino.py: ## @@ -99,11 +101,20 @@ def get_conn(self) -> Connection: elif db.password: auth = trino.auth.BasicAut

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
utkarsharma2 commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399638492 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
utkarsharma2 commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399638492 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
potiuk commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399643942 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more c

Re: [PR] Improve ownership fixing for Breeze [airflow]

2023-11-20 Thread via GitHub
utkarsharma2 commented on code in PR #35759: URL: https://github.com/apache/airflow/pull/35759#discussion_r1399657023 ## scripts/in_container/run_fix_ownership.py: ## @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or

Re: [PR] Remove backcompat inheritance for DbApiHook [airflow]

2023-11-20 Thread via GitHub
Taragolis commented on code in PR #35754: URL: https://github.com/apache/airflow/pull/35754#discussion_r1399661233 ## airflow/providers/trino/hooks/trino.py: ## @@ -99,11 +101,20 @@ def get_conn(self) -> Connection: elif db.password: auth = trino.auth.Basic

Re: [PR] Remove backcompat inheritance for DbApiHook [airflow]

2023-11-20 Thread via GitHub
Taragolis commented on code in PR #35754: URL: https://github.com/apache/airflow/pull/35754#discussion_r1399669338 ## airflow/providers/trino/hooks/trino.py: ## @@ -99,11 +101,20 @@ def get_conn(self) -> Connection: elif db.password: auth = trino.auth.Basic

Re: [PR] Stop logo clashing when pinwheel animates [airflow]

2023-11-20 Thread via GitHub
jscheffl commented on PR #35564: URL: https://github.com/apache/airflow/pull/35564#issuecomment-1819692172 > > > > I agree with Ryan, this was mainly an Easter Egg. I don't know if we want to mess with the logo too much. Can we double check everywhere else we use the Airflow SVG? > > >

Re: [PR] Remove backcompat inheritance for DbApiHook [airflow]

2023-11-20 Thread via GitHub
potiuk commented on code in PR #35754: URL: https://github.com/apache/airflow/pull/35754#discussion_r1399680020 ## airflow/providers/trino/hooks/trino.py: ## @@ -99,11 +101,20 @@ def get_conn(self) -> Connection: elif db.password: auth = trino.auth.BasicAut

Re: [PR] Adding connection-backed Kubernetes Secrets to KubernetesPodOperator [airflow]

2023-11-20 Thread via GitHub
NotYuki commented on code in PR #33680: URL: https://github.com/apache/airflow/pull/33680#discussion_r1399693498 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -706,10 +715,51 @@ def post_complete_action(self, *, pod, remote_pod, **kwargs): remote_pod

Re: [PR] Remove usage of deprecated method from BigQueryToBigQueryOperator [airflow]

2023-11-20 Thread via GitHub
potiuk merged PR #35605: URL: https://github.com/apache/airflow/pull/35605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Remove usage of deprecated method from BigQueryToBigQueryOperator (#35605)

2023-11-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9207e7d5e5 Remove usage of deprecated method from

Re: [PR] allow multiple elements in impersonation chain [airflow]

2023-11-20 Thread via GitHub
melugoyal commented on PR #35694: URL: https://github.com/apache/airflow/pull/35694#issuecomment-1819765823 @VladaZakharova imo the unit tests should be sufficient, given it's just extending the current functionality already in the provider -- This is an automated message from the Apache

Re: [I] Setting a task to run state doesn't do nothing [airflow]

2023-11-20 Thread via GitHub
jscheffl commented on issue #35729: URL: https://github.com/apache/airflow/issues/35729#issuecomment-1819781321 I think this setting of state is an administrative "overright" of the original status and can be used in any kind of troubleshooting - but you need to know what you are doing. It

Re: [I] Create an `OpsgenieNotifier` [airflow]

2023-11-20 Thread via GitHub
potiuk closed issue #35516: Create an `OpsgenieNotifier` URL: https://github.com/apache/airflow/issues/35516 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] feature(providers): added `OpsgenieNotifier` [airflow]

2023-11-20 Thread via GitHub
boring-cyborg[bot] commented on PR #35530: URL: https://github.com/apache/airflow/pull/35530#issuecomment-1819782883 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

(airflow) branch main updated: feature(providers): added `OpsgenieNotifier` (#35530)

2023-11-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2d01cfd4a8 feature(providers): added `OpsgenieNotif

Re: [PR] feature(providers): added `OpsgenieNotifier` [airflow]

2023-11-20 Thread via GitHub
potiuk merged PR #35530: URL: https://github.com/apache/airflow/pull/35530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[PR] Rename --aditional-extras flag to --aditional-airflow-extras [airflow]

2023-11-20 Thread via GitHub
potiuk opened a new pull request, #35760: URL: https://github.com/apache/airflow/pull/35760 The previous flag was wrongly named so it was converted to ADDITIONAL_EXTRAS ARG rather than ADDITIONA_AIRFLOW_EXTRAS. --- **^ Add meaningful description above**

[PR] Set mark_end_on_close after set_context [airflow]

2023-11-20 Thread via GitHub
dstandish opened a new pull request, #35761: URL: https://github.com/apache/airflow/pull/35761 In ES task handler, set_context applies its own logic for mark_end_on_close, so it we must set the attr after for our override to persist. -- This is an automated message from the Apache Git

[PR] Remove elasticsearch constant USE_PER_RUN_LOG_ID [airflow]

2023-11-20 Thread via GitHub
dstandish opened a new pull request, #35762: URL: https://github.com/apache/airflow/pull/35762 Now that the min provider version is 2.5, the constant is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

  1   2   >