charlesccychen commented on a change in pull request #4136: [BEAM-3184] Added
ProxyInfoFromEnvironmentVar() & GetNewHttp() methods for GCS
URL: https://github.com/apache/beam/pull/4136#discussion_r153152127
##########
File path: sdks/python/apache_beam/io/gcp/gcsio.py
##########
@@ -87,6 +87,50 @@
MAX_BATCH_OPERATION_SIZE = 100
+def ProxyInfoFromEnvironmentVar(proxy_env_var):
+ """Reads proxy info from the environment and converts to httplib2.ProxyInfo.
+ Args:
+ proxy_env_var: Environment variable string to read, such as http_proxy or
+ https_proxy.
+ Returns:
+ httplib2.ProxyInfo constructed from the environment string.
+ """
+ proxy_url = os.environ.get(proxy_env_var)
+ if not proxy_url or not proxy_env_var.lower().startswith('http'):
+ return httplib2.ProxyInfo(httplib2.socks.PROXY_TYPE_HTTP, None, 0)
+ proxy_protocol = proxy_env_var.lower().split('_')[0]
+ if not proxy_url.lower().startswith('http'):
+ # proxy_info_from_url requires a protocol, which is always http or https.
+ proxy_url = proxy_protocol + '://' + proxy_url
+ return httplib2.proxy_info_from_url(proxy_url, method=proxy_protocol)
+
+def GetNewHttp(http_class=httplib2.Http, **kwargs):
Review comment:
<!--new_thread; commit:a648a0224e9c7cc59c6d65ae14b03e07ee1e2171;
resolved:0-->
http_class and kwargs are never used in any call to `GetNewHttp`. Please
remove these arguments, as they are not necessary.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
