[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 Adolfo Rodríguez Taboada changed: What|Removed |Added Status|NEW |Needs Signoff --- Comment #16 from Adolfo Rodríguez Taboada --- Thanks for the feedback! I made the changes you suggested and some extra ones: * The system preference is HttpForwardProxy * The new column's name is do_not_use_proxy * The "Do not use proxy" option only appears when HttpForwardProxy is configured and the type is zed (z39.50) * Added HttpForwardProxy in installer/data/mysql/mandatory/sysprefs.sql * Added do_not_use_proxy in the definition of z3950servers installer/data/mysql/kohastructuresql * Changed the regular expression in Breeding.pm -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 --- Comment #15 from Adolfo Rodríguez Taboada --- Created attachment 165835 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165835=edit bug 12620 Allows to search in z39.50 servers using a proxy In some environments it is necessary to use a proxy to search in z39.50 servers wether it's because it doesn't have access to Internet or that the z39.50 ports are blocked. This commit allows to configure a proxy in a new system preference HttpForwardProxy. Sometimes some of the z39.50 servers don't work with the forward proxy, so we have added a column at the z39.50 server level so you can specify which ones won't use the proxy (z3950servers.do_not_use_proxy) To use the proxy, the preference must be configured and the "Do not use proxy" checkbox (z3950servers.do_not_use_proxy) in the z39.50 must be unchecked. The option "Do not use proxy" in the z39.50 server only appears when the system preference is configured. Test plan: 1 Check searching agains z39.50 No results. 2 Apply patch, restart services 3 Configure system preference HttpForwardProxy 4 Check searching agains z39.50 and see that you get results. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 --- Comment #14 from David Cook --- Firstly, thanks for working on this. It hasn't been a priority for me, so it's cool to see someone else take up the thread. Secondly, I'm not sure if you're still working on this, but if you are ready to submit, I'd have to mark this as "Failed QA". 1. You'll need to add the syspref to ./installer/data/mysql/mandatory/sysprefs.sql as well 2. You'll want to use camel case instead of snake case. While there are about 7 snake case there are 827 camel case sysprefs. 3. I'd suggest using HttpForwardProxy as the system preference name just to be as explicit as possible 4. I think that we'd only want to show the "Do not use proxy" on z3950servers.tt *if* HttpForwardProxy is defined. Otherwise, it's going to be meaningless/confusing for 99.99% of users 5. Instead of "not_proxy" we might want to say "do_not_use_proxy" or "no_foward_proxy". They're more verbose, but they're also more descriptive. 6. In Breeding.pm, technically I don't think you'd want to use the "g" option, and you should preface "http" with a ^ so that it matches the start of the string. Your regex would probably be fine in the majority of cases, but I think it wouldn't hurt to be more specific. -- Otherwise I think all good -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 Adolfo Rodríguez Taboada changed: What|Removed |Added CC||adolfo.rodrig...@xercode.es --- Comment #13 from Adolfo Rodríguez Taboada --- Created attachment 165452 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=165452=edit Allows to search in z39.50 servers using a proxy In some environments it is necessary to use a proxy to search in z39.50 servers wether it's because it doesn't have access to Internet or that the z39.50 ports are blocked. This commit allows to configure a proxy in a new system preference HTTPS_Proxy. Sometimes some of the z39.50 servers don't work with the proxy, so we have added a column at server level so you can specify which ones won't use the proxy. To use the proxy the preference must be configured and the "Do not use proxy" checkbox in the z39.50 must be unchecked. Test plan 1 Check searching agains z39.50 No results. 2 Apply patch, restart services 3 Configure system preference HTTPS_Proxy 4 Check searching agains z39.50 and see that you get results. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620
--- Comment #12 from David Cook ---
Actually for readability we could change
'connect:squid:3128,tcp:lx2.loc.gov:210' to
'connect:squid:3128,lx2.loc.gov:210'.
To get this working for Koha...
We would need to change our syntax in C4::Breeding::_create_connection to use
'$obj->connect($host)' instead of '$obj->connect( $server->{host},
$server->{port} )'.
We would also need some way to specify the HTTP Proxy.
Option 1:
Add a http_proxy field (or http_proxy_hostname and http_proxy_port fields) to
the z3950servers table.
Option 2:
Specify a http_proxy in the koha-conf.xml.
This might be a better option, since it's more centralized. Plus, there might
be other parts of Koha that would benefit from being able to go through a HTTP
proxy. (For example OverDrive integration?)
--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620
--- Comment #11 from David Cook ---
Adam inspired me to test this, and I got a proof-of-concept working in Docker
using a Squid proxy.
Here are the steps if someone wants to reproduce my efforts:
Windows:
1. docker pull sameersbn/squid:3.5.27-2
2. docker run --name squid --publish 3128:3128 sameersbn/squid:3.5.27-2
3. docker cp squid:/etc/squid/squid.conf ./squid.conf.default
4. cp ./squid.conf.default ./squid.conf
5. sed -i 's/http_access deny all/http_access allow all/' ./squid.conf
6. sed -i 's/http_access deny CONNECT/http_access allow CONNECT/'
./squid.conf
7. docker stop squid
8. docker rm squid
9. docker run --name squid --publish 3128:3128 --volume
"/$(PWD)/squid.conf:/etc/squid/squid.conf" --net koha_kohanet
sameersbn/squid:3.5.27-2
Koha container:
1. http_proxy=squid:3128 curl -v https://www.google.com
2a. yaz-client connect:squid:3128,tcp:lx2.loc.gov:210
2b. base LCDB
2c. find test
2d. show 1
3a. Use ZOOM in a Perl script to connect to Squid proxy
3b. This works (shows 1 results connecting to LOC LCDB database and
searching for as title of ‘test’)
3c. Script included below*
*
#!/usr/bin/perl
use ZOOM;
my $conn = new ZOOM::Connection('connect:squid:3128,tcp:lx2.loc.gov:210');
$conn->option( databaseName => 'LCDB' );
my $rs = $conn->search_pqf('@attr 1=4 test');
my $n = $rs->size();
warn $n;
--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 --- Comment #10 from David Cook --- I've heard back from Adam Dickmeiss at Indexdata, and he said "you should be able to use [the connect syntax] for most client software of YAZ". So that's interesting! I'm tempted to do some testing, and build this functionality into Koha. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 --- Comment #9 from David Cook --- Note though that this proxying of Z39.50 via a HTTP proxy will only work if your HTTP proxy allows connections to Z39.50 ports (like 210, 7090) and doesn't restrict them to 80 and 443. But still... this has been very illuminating. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620
--- Comment #8 from David Cook ---
(In reply to Galen Charlton from comment #2)
> Depending on the nature of the proxy involved, it may not be necessary to
> use YAZProxy. In particular, the YAZ toolkit supports a special connection
> syntax to direct Z39.50 and SRU traffic through a proxy.
>
> For example, suppose you have a Squid proxy running on localhost and
> listening on port 210, and further suppose that the proxy is configured to
> allow connecting to the target Z39.50 server(s). In that case, with
> yaz-client you can do something like this:
>
> yaz-client connect:localhost:3128,tcp:lx2.loc.gov:210/LCDB
>
> and that will get you a proxied Z39.50 connection to the Library of Congress.
It took a bit of digging but I found some information about that syntax that
Galen mentioned:
https://github.com/indexdata/yaz/blob/master/NEWS#L2365
So in that case you'd be doing "yaz-client
connect:EDUPROXY:PROXYPORT,tcp:lx2.loc.gov:210/LCDB".
Now ZOOM doesn't explicitly mention support for this syntax
(https://metacpan.org/pod/distribution/Net-Z3950-ZOOM/lib/ZOOM.pod). Of course,
neither does yaz-client (https://linux.die.net/man/1/yaz-client). As I said,
the documentation leaves something to be desired.
I'm waiting to hear back from Adam Dickmeiss about that.
But looking at C4::Breeding::_create_connection... it looks like Koha would
need a little patch to change how we pass in the connection string.
Instead of '$obj->connect( $server->{host}, $server->{port} )', we'd want to
use something like '$obj->connect($host)' where $host is constructed based on
whether or not there is a proxy. (See
https://metacpan.org/pod/distribution/Net-Z3950-ZOOM/lib/ZOOM.pod#new())
I could keep digging through source code but I think it'll just be easier to
wait for Adam's response.
--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 --- Comment #7 from David Cook --- It looks like the "proxy" connect option expects the target to be a Z39.50 proxy. However... maybe you could tunnel Z39.50 through HTTP...depending on the proxy... https://en.wikipedia.org/wiki/HTTP_tunnel https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT https://datatracker.ietf.org/doc/html/rfc7231#section-4.3.6 It looks like the YAZ toolkit added something for that in 5.10.0 in 2015: https://github.com/indexdata/yaz/blob/master/NEWS#L457 https://github.com/indexdata/yaz/commit/9021ba4aa6fcdbdd177073d44edd03a86d70198f I'm finding the YAZ documentation to be quite poor... so I'm going to reach out to Indexdata and see what they have to say on the topic. This might actually be possible... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 David Cook changed: What|Removed |Added CC||dc...@prosentient.com.au --- Comment #6 from David Cook --- This is an interesting one, and Dan is in Australia as well, so I'm extra interested. I'd need to know more about the network ecosystem at the school, but it sounds like all outbound network traffic is blocked, unless it uses a HTTP forward proxy. Of course, that means that they'll be restricted to only using HTTP (or maybe just ports 80 and 443 depending on the type of proxy). Since Z39.50 tends to use ports 210 or 7090, that's not going to be an option. SRU could be an option, since it uses HTTP, but the SCIS website seems to suggest it only has Z39.50 for programmatic access (https://www.scisdata.com/products/scis-data/). Looking at https://metacpan.org/pod/ZOOM and https://software.indexdata.com/yaz/doc/zoom.html, there is a proxy option, so if it's not a HTTP forward proxy and is just a TCP proxy... then that could be investigated as an option. Happy to chat with Dan in any case. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 12620] Proxy Add-on for Koha z39.50/SRU servers
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12620 Katrin Fischer changed: What|Removed |Added Summary|Proxy Add-on for Koha |Proxy Add-on for Koha |z39.50 |z39.50/SRU servers -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
