[ https://issues.apache.org/jira/browse/AIRFLOW-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bjorn Olsen updated AIRFLOW-6072: --------------------------------- Description: The boto3 connection used by aws_hook does not respect outbound http_proxy settings (even if these are set in system wide). The way to configure a proxy is to pass a botocore.config.Config object to boto3 when creating a client (according to this SO post). [https://stackoverflow.com/questions/33480108/how-do-you-use-an-http-https-proxy-with-boto3] While the aws_hook get_client_type() method is used extensively by AWS Operators, the "config" argument is not used by any operator. Adding a check to aws_hook for "config" in the "extra_config" of the Airflow Connection, could allow us to pass kwargs there that build the Config object automatically by the hook. Otherwise we have to update every AWS Operator to also take a "config" parameter. To set an outbound proxy is then as simple as adding this to your extra_config: # { .. , # "config":{"proxies": { # "http": "http://myproxy:8080", # "https": "http://myproxy:8080"}}, # ..} was: The boto3 connection used by aws_hook does not respect outbound http_proxy settings (even if these are set in system wide). The way to configure a proxy is to pass a botocore.config.Config object to boto3 when creating a client (according to this SO post). [https://stackoverflow.com/questions/33480108/how-do-you-use-an-http-https-proxy-with-boto3] While the aws_hook get_client_type() method is used extensively by AWS Operators, the "config" argument is not used by any operator. Adding a check to aws_hook for "config" in the "extra_config" of the Airflow Connection, could allow us to pass kwargs there that build the Config object automatically by the hook is created. Otherwise we have to update every AWS Operator to also take a "config" parameter. > aws_hook: Ability to set outbound proxy > --------------------------------------- > > Key: AIRFLOW-6072 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6072 > Project: Apache Airflow > Issue Type: Improvement > Components: aws > Affects Versions: 1.10.6 > Reporter: Bjorn Olsen > Assignee: Bjorn Olsen > Priority: Minor > > The boto3 connection used by aws_hook does not respect outbound http_proxy > settings (even if these are set in system wide). > > The way to configure a proxy is to pass a botocore.config.Config object to > boto3 when creating a client (according to this SO post). > [https://stackoverflow.com/questions/33480108/how-do-you-use-an-http-https-proxy-with-boto3] > While the aws_hook get_client_type() method is used extensively by AWS > Operators, the "config" argument is not used by any operator. > Adding a check to aws_hook for "config" in the "extra_config" of the Airflow > Connection, could allow us to pass kwargs there that build the Config object > automatically by the hook. > Otherwise we have to update every AWS Operator to also take a "config" > parameter. > > To set an outbound proxy is then as simple as adding this to your > extra_config: > # { .. , > # "config":{"proxies": { > # "http": "http://myproxy:8080", > # "https": "http://myproxy:8080"}}, > # ..} -- This message was sent by Atlassian Jira (v8.3.4#803005)