Re: --connect-timeout option and system timeout
Denis Solovyov <[EMAIL PROTECTED]> writes: > Is it true or false that if --connect-timeout is set to a value > larger than timeout implemented by system libraries, it will make no > sense because system timeout will take precedence (i.e. it will > happen earlier than wget's internal timeout)? True. > (I believe that it will be the same answer for --dns-timeout.) And also true.
--connect-timeout option and system timeout
Is it true or false that if --connect-timeout is set to a value larger than timeout implemented by system libraries, it will make no sense because system timeout will take precedence (i.e. it will happen earlier than wget's internal timeout)? (I believe that it will be the same answer for --dns-timeout.) wget 1.10.2 under Linux 2.4. Thank you. Denis Solovyov
Connect timeout
Here is the much awaited connect timeout patch. Although the alarm()
implementation should be the simplest and most portable one, there is
still a portability catch: longjumping out of a signal handler will on
some OS'es leave the signal blocked. You must either explicitly
unblock it using sigblock or use sigsetjmp/siglongjmp which save the
signal set in the first place.
To turn this on, configure checks for the availability of signals (all
Unix systems) and the presence of either sigsetjmp (SYSV/POSIX) or
sigblock (BSD). I believe that should cover virtually all Unix
systems in use today.
For Windows we'll have to write a separate connect_with_timeout,
probably in mswindows.c. That one should use whatever is most
portable under Windows, and is left as an excercise for a
Windows-savvy contributor.
ChangeLog:
2002-04-12 Hrvoje Niksic <[EMAIL PROTECTED]>
* configure.in: Check for . Check for sigsetjmp and
sigblock.
src/ChangeLog:
2002-04-12 Hrvoje Niksic <[EMAIL PROTECTED]>
* connect.c (connect_with_timeout): New function.
(connect_to_one): Use it.
* config.h.in: Add stubs for HAVE_SIGSETJMP, HAVE_SIGBLOCK, and
HAVE_SETJMP_H.
Index: configure.in
===
RCS file: /pack/anoncvs/wget/configure.in,v
retrieving revision 1.32
diff -u -r1.32 configure.in
--- configure.in2001/12/12 08:30:03 1.32
+++ configure.in2002/04/12 01:12:24
@@ -156,7 +156,8 @@
dnl Checks for headers
dnl
AC_CHECK_HEADERS(string.h stdarg.h unistd.h sys/time.h utime.h sys/utime.h pwd.h)
-AC_CHECK_HEADERS(termios.h sys/ioctl.h sys/select.h sys/utsname.h signal.h)
+AC_CHECK_HEADERS(termios.h sys/ioctl.h sys/select.h sys/utsname.h)
+AC_CHECK_HEADERS(signal.h setjmp.h)
AC_HEADER_TIME
dnl
@@ -174,8 +175,8 @@
AC_FUNC_ALLOCA
AC_FUNC_MMAP
AC_CHECK_FUNCS(strdup strstr strcasecmp strncasecmp strpbrk memmove)
-AC_CHECK_FUNCS(gettimeofday mktime strptime)
-AC_CHECK_FUNCS(strerror snprintf vsnprintf select signal symlink access isatty)
+AC_CHECK_FUNCS(gettimeofday mktime strptime strerror snprintf vsnprintf)
+AC_CHECK_FUNCS(select sigblock sigsetjmp signal symlink access isatty)
AC_CHECK_FUNCS(uname gethostname usleep)
dnl
Index: src/config.h.in
===
RCS file: /pack/anoncvs/wget/src/config.h.in,v
retrieving revision 1.20
diff -u -r1.20 config.h.in
--- src/config.h.in 2001/12/06 10:45:27 1.20
+++ src/config.h.in 2002/04/12 01:12:24
@@ -165,6 +165,12 @@
/* Define if you have the signal function. */
#undef HAVE_SIGNAL
+/* Define if you have the sigsetjmp function. */
+#undef HAVE_SIGSETJMP
+
+/* Define if you have the sigblock function. */
+#undef HAVE_SIGBLOCK
+
/* Define if you have the gettext function. */
#undef HAVE_GETTEXT
@@ -200,6 +206,9 @@
/* Define if you have the header file. */
#undef HAVE_SIGNAL_H
+
+/* Define if you have the header file. */
+#undef HAVE_SETJMP_H
/* Define if you have the header file. */
#undef HAVE_LIBINTL_H
Index: src/connect.c
===
RCS file: /pack/anoncvs/wget/src/connect.c,v
retrieving revision 1.13
diff -u -r1.13 connect.c
--- src/connect.c 2002/01/25 03:34:23 1.13
+++ src/connect.c 2002/04/12 01:12:24
@@ -47,6 +47,28 @@
# include
#endif /* HAVE_SYS_SELECT_H */
+/* To implement connect with timeout, we need signal and either
+ sigsetjmp or sigblock. */
+#undef UNIX_CONNECT_TIMEOUT
+#ifdef HAVE_SIGNAL_H
+# include
+#endif
+#ifdef HAVE_SETJMP_H
+# include
+#endif
+/* If sigsetjmp is a macro, configure won't pick it up. */
+#ifdef sigsetjmp
+# define HAVE_SIGSETJMP
+#endif
+#ifdef HAVE_SIGNAL
+# ifdef HAVE_SIGSETJMP
+# define UNIX_CONNECT_TIMEOUT
+# endif
+# ifdef HAVE_SIGBLOCK
+# define UNIX_CONNECT_TIMEOUT
+# endif
+#endif
+
#include "wget.h"
#include "host.h"
#include "connect.h"
@@ -84,7 +106,82 @@
address_list_release (al);
bind_address_resolved = 1;
}
+
+#ifndef UNIX_CONNECT_TIMEOUT
+static int
+connect_with_timeout (int fd, const struct sockaddr *addr, int addrlen,
+ int timeout)
+{
+ return connect (fd, addr, addrlen);
+}
+#else /* UNIX_CONNECT_TIMEOUT */
+/* Implementation of connect with timeout. */
+
+#ifdef HAVE_SIGSETJMP
+#define SETJMP(env) sigsetjmp (env, 1)
+
+static sigjmp_buf abort_connect_env;
+
+RETSIGTYPE
+abort_connect (int ignored)
+{
+ siglongjmp (abort_connect_env, -1);
+}
+#else /* not HAVE_SIGSETJMP */
+#define SETJMP(env) setjmp (env)
+
+static jmp_buf abort_connect_env;
+
+RETSIGTYPE
+abort_connect (int ignored)
+{
+ /* We don't have siglongjmp to preserve the set of blocked signals;
+ if we longjumped out of the handler at this point, SIGALRM would
+ remain blocked. We must unblock it manually. */
+ int mask = siggetmask ();
+ mask &= ~sigmask
Re: connect timeout
Daniel Stenberg wrote: > > On Wed, 7 Nov 2001, Hack Kampbjørn wrote: > > > Well, Daniel Stenberg maybe you should try to get your cURL > > implementation accepted. > > AFAIK, there's no one around to "accept" any fixes or contributions right Well, you got my point 8-) > now. I've even emailed Hrvoje with my own views, that he should open up the > CVS to let more people have write access. I'd suggest that we cough up a few > people that would be willing to shoulder that responsibility and that they > get in touch with Hrvoje to sort things out. I'm not talking mutiny, I'm > talking joining up behind the team leader, showing and offering support. Well, is somebody qualified to and interested in, actively mantaining wget. As you say a fork should not be necessary, Hrvoje is surely interested in a well mantained wget. Last time Dan succeeded in tracking him down and becoming a mantainer. It should be possible again if somebody is interested in the job. Hopefully there would be more than three committers so it's more protected against changing work situations. > > Anyway, if I would write down a patch for this, would the -T timeout value be > the one to use even for connects? "set the read timeout to SECONDS" doesn't > quite sound like what we want here, we want a "set the connect timeout to > SECONDS"... but would we want to introduce yet another option? > > > It's bad when other packages maintainers are more active on the list that > > Wget's 8-( > > The fact that I maintain curl, or that I am active here too is really not the > problem. The problem here is the lack of a Hrvoje stand-in. Sorry, I didn't mean it's bad that you're active here. Your experience implementing similar functionality in curl is really appreciated > > I'm active in numerous other free/open projects as well. Now, that you say it, I've seen your 'unix is cool' signature somewhere else too 8-) > > -- > Daniel Stenberg - http://daniel.haxx.se - +46-705-44 31 77 >ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol -- Med venlig hilsen / Kind regards Hack Kampbjørn
Re: connect timeout
On Wed, 7 Nov 2001, Hack Kampbjørn wrote: > Well, Daniel Stenberg maybe you should try to get your cURL > implementation accepted. AFAIK, there's no one around to "accept" any fixes or contributions right now. I've even emailed Hrvoje with my own views, that he should open up the CVS to let more people have write access. I'd suggest that we cough up a few people that would be willing to shoulder that responsibility and that they get in touch with Hrvoje to sort things out. I'm not talking mutiny, I'm talking joining up behind the team leader, showing and offering support. Anyway, if I would write down a patch for this, would the -T timeout value be the one to use even for connects? "set the read timeout to SECONDS" doesn't quite sound like what we want here, we want a "set the connect timeout to SECONDS"... but would we want to introduce yet another option? > It's bad when other packages maintainers are more active on the list that > Wget's 8-( The fact that I maintain curl, or that I am active here too is really not the problem. The problem here is the lack of a Hrvoje stand-in. I'm active in numerous other free/open projects as well. -- Daniel Stenberg - http://daniel.haxx.se - +46-705-44 31 77 ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: connect timeout
Nic Ferrier wrote:
>
> >The official web-site is http://wget.sunsite.dk/
> >Yes, there should be a link from the GNU site.
> >[List] how can it be added ?
>
> sign the project up on savannah (http://savannah.gnu.org).
>
> That will provide you with a nice management interface (based on CVS)
> for changing the wget tree.
>
> >$ wget --help
> >GNU Wget 1.7, a non-interactive network retriever.
> >Usage: wget [OPTION]... [URL]...
> >Download:
> > -t, --tries=3DNUMBER set number of retries to NUMBER (0
> > unlimits).
> > -T, --timeout=3DSECONDSset the read timeout to SECONDS.
> > -w, --wait=3DSECONDS wait SECONDS between retrievals.
> > --waitretry=3DSECONDS wait 1...SECONDS between retries of a
> > retrieval.
> >Which of this were you using ?
>
> I tried sveral things, including:
>
>wget -t 2 -T 10 -w 1
>
> But, be honest, is the -T option actually a *connect* timeout?
You're right it's not 8-(
$ wget -d -T 5 http://192.168.1.254/
DEBUG output created by Wget 1.7 on cygwin.
parseurl ("http://192.168.1.254/";) -> host 192.168.1.254 -> opath ->
dir -> file -> ndir
newpath: /
--22:23:46-- http://192.168.1.254/
=> `index.html'
Connecting to 192.168.1.254:80...
connect: Attempt to connect timed out without establishing a connection
Closing fd 3
Retrying.
--22:24:08-- http://192.168.1.254/
(try: 2) => `index.html'
Connecting to 192.168.1.254:80...
[hack@DUR0N2000 webs]$ wget -d -T 5 http://192.168.1.254/
DEBUG output created by Wget 1.7 on cygwin.
parseurl ("http://192.168.1.254/";) -> host 192.168.1.254 -> opath ->
dir -> file -> ndir
newpath: /
--22:24:19-- http://192.168.1.254/
=> `index.html'
Connecting to 192.168.1.254:80...
[...]
$ wget -d -T 5 http://hostname/
DEBUG output created by Wget 1.7-dev on linux-gnu.
parseurl ("http://hostname/";) -> host hostname -> opath -> dir ->
file -> ndir
newpath: /
--22:36:48-- http://hostname/
=> `index.html'
Connecting to hostname:80... Caching hostname <-> 192.168.1.254
connect: Connection timed out
Closing fd 3
Retrying.
--22:39:58-- http://hostname/
(try: 2) => `index.html'
Connecting to hostname:80... Found hostname in host_name_address_map:
192.168.1.254
Note: output edited "hostname" is a host that doesn't answer on port 80.
Two different systems on two different networks, that might explain the
difference in timeout times.
Well, Daniel Stenberg maybe you should try to get your cURL
implementation accepted. It's bad when other packages maintainers are
more active on the list that Wget's 8-(
>
> >And please the next time send bugreports including debug output (wget
> >-d =2E..)
>
> I don't think it would do you much good in this case... but I can send
> you one if you want.
Likely not, but it would include the Wget version and if that's not 1.7
then the standard recommendation would be to update. You'll be surpised
how many bug-reports there is related to older version like 1.5.3 or
even a couple to 1.4.5 8-)
>
> Nic
--
Med venlig hilsen / Kind regards
Hack Kampbjørn
Re: connect timeout
>The official web-site is http://wget.sunsite.dk/ >Yes, there should be a link from the GNU site. >[List] how can it be added ? sign the project up on savannah (http://savannah.gnu.org). That will provide you with a nice management interface (based on CVS) for changing the wget tree. >$ wget --help >GNU Wget 1.7, a non-interactive network retriever. >Usage: wget [OPTION]... [URL]... >Download: > -t, --tries=3DNUMBER set number of retries to NUMBER (0 > unlimits). > -T, --timeout=3DSECONDSset the read timeout to SECONDS. > -w, --wait=3DSECONDS wait SECONDS between retrievals. > --waitretry=3DSECONDS wait 1...SECONDS between retries of a > retrieval. >Which of this were you using ? I tried sveral things, including: wget -t 2 -T 10 -w 1 But, be honest, is the -T option actually a *connect* timeout? >And please the next time send bugreports including debug output (wget >-d =2E..) I don't think it would do you much good in this case... but I can send you one if you want. Nic
Re: connect timeout
Nic Ferrier wrote: > > Sorry if you're already aware of this... I couldn't find the archives of > this list at GNU. Maybe you should put a link on the page: > http://www.gnu.org/software/wget/ The official web-site is http://wget.sunsite.dk/ Yes, there should be a link from the GNU site. [List] how can it be added ? > > I've discovered that wget doesn't do connection timeouts. That is if the > host it is trying to connect to cannot be reached for some reason then wget > simply hangs. > > I expected wget to return after T seconds after specifying the timeout > option on the command line but it didn't. > > No control of connect timeouts is a serious weakness in a tool designed to > be used for batched downloads... I've had to swap wget for curl for the > particular task I'm working on (which is a pity because in all other > respects I like wget and want to support GNU projects). $ wget --help GNU Wget 1.7, a non-interactive network retriever. Usage: wget [OPTION]... [URL]... [...] Download: [...] -t, --tries=NUMBER set number of retries to NUMBER (0 unlimits). [...] -T, --timeout=SECONDSset the read timeout to SECONDS. -w, --wait=SECONDS wait SECONDS between retrievals. --waitretry=SECONDS wait 1...SECONDS between retries of a retrieval. [...] Which of this were you using ? And please the next time send bugreports including debug output (wget -d ...) > > Nic Ferrier -- Med venlig hilsen / Kind regards Hack Kampbjørn
Re: connect timeout
On Wed, 7 Nov 2001, Nic Ferrier wrote:
> I've discovered that wget doesn't do connection timeouts. That is if the
> host it is trying to connect to cannot be reached for some reason then
> wget simply hangs.
>
> I expected wget to return after T seconds after specifying the timeout
> option on the command line but it didn't.
>
> No control of connect timeouts is a serious weakness in a tool designed
> to be used for batched downloads... I've had to swap wget for curl for
> the particular task I'm working on (which is a pity because in all other
> respects I like wget and want to support GNU projects).
Well.
As author of those parts of the curl code, I'm willing to donate code to wget
that would bring this functionality. Should anyone want it.
("donate" in this context would just be to hand over the copyright, as curl
is already completely free and open software)
--
Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/
connect timeout
Sorry if you're already aware of this... I couldn't find the archives of this list at GNU. Maybe you should put a link on the page: http://www.gnu.org/software/wget/ I've discovered that wget doesn't do connection timeouts. That is if the host it is trying to connect to cannot be reached for some reason then wget simply hangs. I expected wget to return after T seconds after specifying the timeout option on the command line but it didn't. No control of connect timeouts is a serious weakness in a tool designed to be used for batched downloads... I've had to swap wget for curl for the particular task I'm working on (which is a pity because in all other respects I like wget and want to support GNU projects). Nic Ferrier
