potiuk opened a new pull request, #45704:
URL: https://github.com/apache/airflow/pull/45704
Follow up after #45690 and #45682
Wee already had protection against example dags not using database, but it
turns out that just calling get_connection() of the BaseHook involves calling
out to secrets manager which - depending on the configuration, providers and
where it is called - might cause external calls, timeout and various side
effects.
This PR adds explicit test for that. As part of the change we also added
`--load-example-dags` and `--load-default-connections` to breeze shell as it
allows to easily test the case where default connections are loaded in the
database.
Note that the "example_bedrock_retrieve_and_generate" explicitly avoided
attempting to load the connections by specifing aws_conn_id to None, because it
was likely causing problems with fetching SSM when get_connection was actually
called during dag parsing, so this aws_conn_id = None would also bypass this
check, but we can't do much about it - at least after this chanege, the
contributor will see failing test with explicit "get_connection() should not be
called during DAG parsing".
That also makes the example dag more of a "real" example as it does not
nullify the connection id and it can use "aws_default" connection to actually
... be a good example. Also it allows to run the example dag as system test for
someone who would like to do it with "aws_default" as a connection id to
connect to their AWS account.
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
