Bug#801136: mpd-sima: Troublesome with proxies
Followup-For: Bug #801136
Package: mpd-sima
Tags: patch, confirmed
Here is a patch, it should merge available HTTPS?_PROXIES variables.
I will patch the debian package if it takes too long to release the next
upstream version including this fix.
I haven't fully tested the solution yet.
Don't hesitate to give it a try, the patch should merge fine on 0.13.
I'll have a look at the packaging part later.
Cheers
Geoff
Le 10/10/2015 15:07, Geoff a écrit :
> Hi Chris,
>
> Thanks very much for your report :)
>
> Two issues then :
>
> [packaging] http proxies env. var should be exposed when using
> systemd/SystV.
>
> [upstream] the http client should actually use them ^^
> The http client bug was forwarded upstream, the issue is
> identified/nearly fixed. Python requests is actually not honoring
> HTTP_PROXY because of the use of the lower level API in the http client
> which in turn is not propagating environment variables…
>
> I'll try to fix this as soon as I can (I'm also the upstream developer).
>
> Thanks again.
> Geoff
>
> The 06/10/2015 17:46, Chris Chiappa wrote :
>> […]
>> Behind a proxy, I've found it very difficult to get mpd-sima to use it
>> for last.fm access. python-requests documentation at
>> http://docs.python-requests.org/en/latest/ says it should pay
>> attention to the HTTP_PROXY and HTTPS_PROXY environment variables. I
>> created /etc/systemd/system/mpd-sima.service.d/override.conf
>> with
>>
>> [Service]
>> Environment="HTTP_PROXY=http://proxy.company.com:80";
>> Environment="HTTPS_PROXY=http://proxy.company.com:80";
>>
>> and restarted mpd-sima. In /proc//environ I can see both
>> environment variables, but from both strace and mpd-sima's log I can
>> see that it's failing to connect to last.fm. I was able to get it to
>> work by hacking fetch_ws in http.py and manually specifying proxies:
>>
>> @Throttle(WAIT_BETWEEN_REQUESTS)
>> def fetch_ws(self, prepreq):
>> proxies = {
>> "http" : "http://proxy.company.com:80";,
>> "https" : "http://proxy.company.com:80";
>> }
>> """fetch from web service"""
>> sess = Session()
>> sess.proxies = proxies
>> resp = sess.send(prepreq, timeout=SOCKET_TIMEOUT)
>> ...
>>
>> but that's obviously not the right way to do it. Not being a python
>> hacker I'm not sure how to tell why the envrironment variables aren't
>> working, but it seems like having some better way of configuring
>> proxies might be desirable anyhow.
>
diff --git a/sima/lib/http.py b/sima/lib/http.py
index 0c1b396..10fae9c 100644
--- a/sima/lib/http.py
+++ b/sima/lib/http.py
@@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
-# Copyright (c) 2014 Jack Kaliko
+# Copyright (c) 2014-2015 Jack Kaliko
# Copyright (c) 2012, 2013 Eric Larson
#
# This program is free software: you can redistribute it and/or modify
@@ -301,7 +301,8 @@ class HttpClient:
def fetch_ws(self, prepreq):
"""fetch from web service"""
sess = Session()
-resp = sess.send(prepreq, timeout=SOCKET_TIMEOUT)
+settings = sess.merge_environment_settings(prepreq.url, {}, None, False, None)
+resp = sess.send(prepreq, timeout=SOCKET_TIMEOUT, **settings)
if resp.status_code == 304:
self.stats.update(etag=self.stats.get('etag')+1)
resp = self.controller.update_cached_response(prepreq, resp)
signature.asc
Description: OpenPGP digital signature
Bug#801136: mpd-sima: Troublesome with proxies
Hi Chris,
Thanks very much for your report :)
Two issues then :
[packaging] http proxies env. var should be exposed when using
systemd/SystV.
[upstream] the http client should actually use them ^^
The http client bug was forwarded upstream, the issue is
identified/nearly fixed. Python requests is actually not honoring
HTTP_PROXY because of the use of the lower level API in the http client
which in turn is not propagating environment variables…
I'll try to fix this as soon as I can (I'm also the upstream developer).
Thanks again.
Geoff
The 06/10/2015 17:46, Chris Chiappa wrote :
> […]
> Behind a proxy, I've found it very difficult to get mpd-sima to use it
> for last.fm access. python-requests documentation at
> http://docs.python-requests.org/en/latest/ says it should pay
> attention to the HTTP_PROXY and HTTPS_PROXY environment variables. I
> created /etc/systemd/system/mpd-sima.service.d/override.conf
> with
>
> [Service]
> Environment="HTTP_PROXY=http://proxy.company.com:80";
> Environment="HTTPS_PROXY=http://proxy.company.com:80";
>
> and restarted mpd-sima. In /proc//environ I can see both
> environment variables, but from both strace and mpd-sima's log I can
> see that it's failing to connect to last.fm. I was able to get it to
> work by hacking fetch_ws in http.py and manually specifying proxies:
>
> @Throttle(WAIT_BETWEEN_REQUESTS)
> def fetch_ws(self, prepreq):
> proxies = {
> "http" : "http://proxy.company.com:80";,
> "https" : "http://proxy.company.com:80";
> }
> """fetch from web service"""
> sess = Session()
> sess.proxies = proxies
> resp = sess.send(prepreq, timeout=SOCKET_TIMEOUT)
> ...
>
> but that's obviously not the right way to do it. Not being a python
> hacker I'm not sure how to tell why the envrironment variables aren't
> working, but it seems like having some better way of configuring
> proxies might be desirable anyhow.
signature.asc
Description: OpenPGP digital signature
Bug#801136: mpd-sima: Troublesome with proxies
Package: mpd-sima
Version: 0.13.1-1
Severity: normal
Behind a proxy, I've found it very difficult to get mpd-sima to use it
for last.fm access. python-requests documentation at
http://docs.python-requests.org/en/latest/ says it should pay
attention to the HTTP_PROXY and HTTPS_PROXY environment variables. I
created /etc/systemd/system/mpd-sima.service.d/override.conf
with
[Service]
Environment="HTTP_PROXY=http://proxy.company.com:80";
Environment="HTTPS_PROXY=http://proxy.company.com:80";
and restarted mpd-sima. In /proc//environ I can see both
environment variables, but from both strace and mpd-sima's log I can
see that it's failing to connect to last.fm. I was able to get it to
work by hacking fetch_ws in http.py and manually specifying proxies:
@Throttle(WAIT_BETWEEN_REQUESTS)
def fetch_ws(self, prepreq):
proxies = {
"http" : "http://proxy.company.com:80";,
"https" : "http://proxy.company.com:80";
}
"""fetch from web service"""
sess = Session()
sess.proxies = proxies
resp = sess.send(prepreq, timeout=SOCKET_TIMEOUT)
...
but that's obviously not the right way to do it. Not being a python
hacker I'm not sure how to tell why the envrironment variables aren't
working, but it seems like having some better way of configuring
proxies might be desirable anyhow.
-- System Information:
Debian Release: stretch/sid
APT prefers unstable
APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 4.1.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages mpd-sima depends on:
ii adduser 3.113+nmu3
ii init-system-helpers 1.23
ii lsb-base 9.20150917
ii python3 3.4.3-6
ii python3-musicpd 0.4.1-1
ii python3-requests 2.7.0-3
pn python3:any
mpd-sima recommends no packages.
Versions of packages mpd-sima suggests:
ii mpd 0.19.10-1+b3
-- Configuration Files:
/etc/default/mpd-sima changed:
START_MPD_SIMA=true
HTTP_PROXY=http://www-proxy.us.oracle.com:80
HTTPS_PROXY=http://www-proxy.us.oracle.com:80
export HTTP_PROXY HTTPS_PROXY
/etc/mpd-sima.cfg [Errno 13] Permission denied: u'/etc/mpd-sima.cfg'
-- no debconf information
