Hello,
I was in real big need of this feature in wget, I'll explain below why...
On 20.03.2012 18:30, Ángel González wrote:
[...]
>> Is it a bug? My original message was about asking what the meaning was
>> of this output...
> I have looked at the patch and it apparently works fine. In fact, there's a
> comparison to not do anything else if libproxy returned direct://
> The only bug seem to be that such message is confusing. It should probably
> have been shown only with --debug.
I think the reason why one would want to use libproxy support would
rather be the possibility to get proxy information from a PAC file.
Imagine the situation inside a corporate environment where such a PAC
file might be several pages long, so setting just one proxy server would
only work for few URLS.
For that, libproxy itself has to be built with support of a pacrunner,
which might be a dependency to webkit-gtk, xulrunner or other mozilla
derivates which all can parse JavaScript. Of course, this is out of
scope here, as far as wget is concerned, it should be just either build
with or without libproxy support. If one wants to explicitely ask
libproxy to use a specific PAC file in order to get the right proxy
settings for the specific URL, the environment variable for the
respective proxy type (http_proxy, https_proxy or ftp_proxy) has to be
set like
"http_proxy=wpad://" or
"http_proxy=pac+http://server/path/to.pac";
and then libproxy will do the rest (see comments 14 to 19 of
http://code.google.com/p/libproxy/issues/detail?id=129 for details). But
in the old variant of the patch it failed on this, because the scheme
"pac+http" is rejected by wget before libproxy gets the chance to see
it. My patch runs libproxy before falling back to the original code (it
should mostly even not need to fall back if libproxy did the job right)
and just pass the proxy setting to wget.
There was one more problem with the old patch, the debug statements used
code which was only built when debug was generally enabled, so I moved
the #ifdef DEBUG_ENABLED statements inside the definition of
debug_logprintf, that way the code is either an empty function, or one
which only logs when we built with debug upport and also request debug
output. At least it can no longer fail on linking...
Would be nice if this feature could be included in the next release,
what do you think? Of course, main credits go to Dominique, I only
re-arranged his patch.
Regards,
Lucian
diff -Naur wget-1.13.4_orig/configure.ac wget-1.13.4/configure.ac
--- wget-1.13.4_orig/configure.ac 2011-09-04 13:58:44.000000000 +0200
+++ wget-1.13.4/configure.ac 2012-04-18 07:42:24.192453012 +0200
@@ -338,6 +338,22 @@
fi
fi
+dnl
+dnl libproxy support
+dnl
+AC_ARG_ENABLE(libproxy,
+ [ --enable-libproxy libproxy support for system wide proxy
configuration])
+if test "${enable_libproxy}" != "no"
+then
+ PKG_CHECK_MODULES([libproxy], [libproxy-1.0], [enable_libproxy=yes],
[enable_libproxy=no])
+fi
+if test "${enable_libproxy}" = "yes"
+then
+ AC_SUBST(libproxy_CFLAGS)
+ AC_SUBST(libproxy_LIBS)
+ AC_DEFINE([HAVE_LIBPROXY], 1, [Define when using libproxy])
+fi
+
dnl **********************************************************************
dnl Checks for IPv6
dnl **********************************************************************
diff -Naur wget-1.13.4_orig/src/Makefile.am wget-1.13.4/src/Makefile.am
--- wget-1.13.4_orig/src/Makefile.am 2011-08-18 13:44:47.000000000 +0200
+++ wget-1.13.4/src/Makefile.am 2012-04-18 07:42:24.192453012 +0200
@@ -37,7 +37,7 @@
# The following line is losing on some versions of make!
DEFS = @DEFS@ -DSYSTEM_WGETRC=\"$(sysconfdir)/wgetrc\"
-DLOCALEDIR=\"$(localedir)\"
-LIBS = @LIBICONV@ @LIBINTL@ @LIBS@ $(LIB_CLOCK_GETTIME)
+LIBS = @LIBICONV@ @LIBINTL@ @LIBS@ $(LIB_CLOCK_GETTIME) @libproxy_LIBS@
EXTRA_DIST = css.l css.c css_.c build_info.c.in
diff -Naur wget-1.13.4_orig/src/log.c wget-1.13.4/src/log.c
--- wget-1.13.4_orig/src/log.c 2011-07-29 15:43:44.000000000 +0200
+++ wget-1.13.4/src/log.c 2012-04-18 15:09:43.489453013 +0200
@@ -41,6 +41,7 @@
#include "utils.h"
#include "log.h"
+
/* 2005-10-25 SMS.
VMS log files are often VFC record format, not stream, so fputs() can
produce multiple records, even when there's no newline terminator in
@@ -506,12 +507,12 @@
while (!done);
}
-#ifdef ENABLE_DEBUG
/* The same as logprintf(), but does anything only if opt.debug is
true. */
void
debug_logprintf (const char *fmt, ...)
{
+#ifdef ENABLE_DEBUG
if (opt.debug)
{
va_list args;
@@ -531,8 +532,8 @@
}
while (!done);
}
-}
#endif /* ENABLE_DEBUG */
+}
/* Open FILE and set up a logging stream. If FILE cannot be opened,
exit with status of 1. */
diff -Naur wget-1.13.4_orig/src/retr.c wget-1.13.4/src/retr.c
--- wget-1.13.4_orig/src/retr.c 2011-08-30 15:47:33.000000000 +0200
+++ wget-1.13.4/src/retr.c 2012-04-18 15:29:11.350453013 +0200
@@ -54,6 +54,10 @@
#include "html-url.h"
#include "iri.h"
+#ifdef HAVE_LIBPROXY
+#include "proxy.h"
+#endif
+
/* Total size of downloaded files. Used to enforce quota. */
SUM_SIZE_INT total_downloaded_bytes;
@@ -1165,7 +1169,36 @@
if (no_proxy_match (u->host, (const char **)opt.no_proxy))
return NULL;
- switch (u->scheme)
+#ifdef HAVE_LIBPROXY
+ pxProxyFactory *pf = px_proxy_factory_new();
+ if (!pf)
+ {
+ debug_logprintf (_("retr.c, getproxy: Allocating memory for libproxy
failed"));
+ return NULL;
+ }
+ int i;
+ char direct[] = "direct://";
+
+ debug_logprintf (_("retr.c, getproxy: asking libproxy which proxy to use for
url '%s'\n"), u->url);
+ char **proxies = px_proxy_factory_get_proxies(pf, u->url);
+ if (proxies[0])
+ {
+ char *check = NULL;
+ asprintf(&check , "%s", proxies[0]);
+ if(strcmp(check ,direct) != 0)
+ {
+ asprintf(&proxy , "%s", proxies[0]);
+ }
+ debug_logprintf (_("retr.c, getproxy: libproxy suggests to use '%s'\n"),
check);
+ }
+ for(i=0;proxies[i];i++) free(proxies[i]);
+ free(proxies);
+ free(pf);
+#endif
+
+ if (!proxy || !*proxy)
+ {
+ switch (u->scheme)
{
case SCHEME_HTTP:
proxy = opt.http_proxy ? opt.http_proxy : getenv ("http_proxy");
@@ -1183,6 +1216,7 @@
}
if (!proxy || !*proxy)
return NULL;
+ }
/* Handle shorthands. `rewritten_storage' is a kludge to allow
getproxy() to return static storage. */
