[
https://issues.apache.org/jira/browse/AIRFLOW-6822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045153#comment-17045153
]
ASF GitHub Bot commented on AIRFLOW-6822:
-----------------------------------------
baolsen commented on pull request #7541: [AIRFLOW-6822] AWS hooks should cache
boto3 client
URL: https://github.com/apache/airflow/pull/7541
---
Issue link: WILL BE INSERTED BY
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN =
JIRA ID<sup>*</sup>
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
<sup>*</sup> For document-only changes commit message can start with
`[AIRFLOW-XXXX]`.
---
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> AWS hooks dont always cache the boto3 client
> --------------------------------------------
>
> Key: AIRFLOW-6822
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6822
> Project: Apache Airflow
> Issue Type: Bug
> Components: aws
> Affects Versions: 1.10.9
> Reporter: Bjorn Olsen
> Assignee: Bjorn Olsen
> Priority: Minor
>
> Implementation of the Amazon AWS hooks (eg S3 hook, Glue hook etc) varies
> with how they call the underlying aws_hook.get_client_type(X) method.
> Most of the time the client that gets returned is cached by the superclass,
> but not always. The client should always be cached for performance reasons -
> creating a client is a time consuming process.
> Example of how to do it (athena.py):
>
> {code:java}
> def get_conn(self):
> """
> check if aws conn exists already or create one and return it
> :return: boto3 session
> """
> if not self.conn:
> self.conn = self.get_client_type('athena')
> return self.conn{code}
>
>
> Example of how not to do it: (s3.py):
>
> {code:java}
> def get_conn(self):
> return self.get_client_type('s3'){code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
