Re: [Bug-wget] wget alpha release 1.14.96-38327
@Noel: Could you please elaborate on the patches that fixes bugs in your bugtracker? @Giuseppe: Do we have any blocking bugs that need fixing? If not, would it be possible to make a release before the end of the year? If anything needs fixing, I'd like to help and ensure a new release ASAP. On Thu, Nov 21, 2013 at 5:14 AM, Ray Satiro raysat...@yahoo.com wrote: hi , SSL autodetection (am I correct that is supposed to be the default?) doesn't seem to be working here in mingw, it tells me: configure: error: --with-ssl=gnutls was given, but GNUTLS is not available. I don't have GNUTLS so I expect it would use openssl. When I explicitly use --with-ssl=openssl that works. pthread lib detected but not included: checking for pthread.h... yes checking for pthread_kill in -lpthread... yes checking for multithread API to use... posix later: gcc -O2 -Wall -o wget.exe cmpt.o connect.o convert.o cookies.o ftp.o css_.o c ss-url.o ftp-basic.o ftp-ls.o hash.o host.o html-parse.o html-url.o http.o init. o log.o main.o netrc.o progress.o ptimer.o recur.o res.o retr.o spider.o url.o w arc.o utils.o exits.o build_info.o version.o ftp-opie.o mswindows.o openssl.o h ttp-ntlm.o ../lib/libgnu.a -liconv -lintl -leay32 -lz -lws2_32 -lssl32 ../lib/libgnu.a(regex.o):regex.c:(.text+0xa4f1): undefined reference to `__imp__ pthread_mutex_init' ../lib/libgnu.a(regex.o):regex.c:(.text+0xa92e): undefined reference to `__imp__ pthread_mutex_destroy' ../lib/libgnu.a(regex.o):regex.c:(.text+0xb146): undefined reference to `__imp__ pthread_mutex_lock' ../lib/libgnu.a(regex.o):regex.c:(.text+0xb214): undefined reference to `__imp__ pthread_mutex_unlock' ../lib/libgnu.a(regex.o):regex.c:(.text+0xb6fe): undefined reference to `__imp__ pthread_mutex_destroy' ../lib/libgnu.a(regex.o):regex.c:(.text+0xb78a): undefined reference to `__imp__ pthread_mutex_lock' ../lib/libgnu.a(regex.o):regex.c:(.text+0xb7d3): undefined reference to `__imp__ pthread_mutex_unlock' When I add -lpthread to LIBS that works. When wget is compiled using current versions of MinGW there is a problem because it will link to functions that do not exist in earlier versions of the crt, = Vista usually. I brought that up on their mailing list earlier this month, http://sourceforge.net/mailarchive/forum.php?thread_name=52796A9F.20905%40users.sourceforge.netforum_name=mingw-users No fix yet but a workaround would be to ignore ftelli64 and fseeki64 in crt. What is the version scheme? The last alpha build I have from earlier this year before 1.14.96-38327 is 1.14.128-something Excuse the weird formatting and lack of quoting I can't find a way to do some things in the new Yahoo mail. Thanks On Saturday, November 2, 2013 8:33 AM, Giuseppe Scrivano gscriv...@gnu.org wrote: Hi, I have just uploaded an alpha release for wget. If no blocking errors are found, then I will make an official release in the coming weeks (that is, wget 1.15). Please test it! :-) http://alpha.gnu.org/gnu/wget/wget-1.14.96-38327.tar.gz http://alpha.gnu.org/gnu/wget/wget-1.14.96-38327.tar.xz signatures, using key C03363F4: http://alpha.gnu.org/gnu/wget/wget-1.14.96-38327.tar.gz.sig http://alpha.gnu.org/gnu/wget/wget-1.14.96-38327.tar.xz.sig Thanks, Giuseppe -- Thanking You, Darshit Shah
[Bug-wget] [PATCH v3] Make wget capable of starting download from a specified position.
This patch adds an option `--start-pos' for specifying starting position
of a download, both for HTTP and FTP. When specified, the newly added
option would override `--continue'. Apart from that, no existing code
should be affected.
Signed-off-by: Yousong Zhou yszhou4t...@gmail.com
---
v2 - v3
Fix a typo and add description text for the new option into the usage
output. Thank Darshit Shah dar...@gmail.com for the suggestions.
v1 - v2
It was kindly pointed out by Darshit Shah dar...@gmail.com that
server support for resuming download is required, so adding this into
doc/wget.texi.
doc/ChangeLog |4
doc/wget.texi | 17 +
src/ChangeLog |9 +
src/ftp.c |2 ++
src/http.c|2 ++
src/init.c|1 +
src/main.c|3 +++
src/options.h |1 +
8 files changed, 39 insertions(+), 0 deletions(-)
From f7266cc18fbea1d07b25c1bd25662a5a71920520 Mon Sep 17 00:00:00 2001
From: Yousong Zhou yszhou4t...@gmail.com
Date: Fri, 20 Dec 2013 23:17:43 +0800
Subject: [PATCH v3] Make wget capable of starting download from a specified position.
This patch adds an option `--start-pos' for specifying starting position
of a download, both for HTTP and FTP. When specified, the newly added
option would override `--continue'. Apart from that, no existing code
should be affected.
Signed-off-by: Yousong Zhou yszhou4t...@gmail.com
---
v2 - v3
Fix a typo and add description text for the new option into the usage
output. Thank Darshit Shah dar...@gmail.com for the suggestions.
v1 - v2
It was kindly pointed out by Darshit Shah dar...@gmail.com that
server support for resuming download is required, so adding this into
doc/wget.texi.
doc/ChangeLog |4
doc/wget.texi | 17 +
src/ChangeLog |9 +
src/ftp.c |2 ++
src/http.c|2 ++
src/init.c|1 +
src/main.c|3 +++
src/options.h |1 +
8 files changed, 39 insertions(+), 0 deletions(-)
diff --git a/doc/ChangeLog b/doc/ChangeLog
index 3b05756..df103c8 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,7 @@
+2013-12-21 Yousong Zhou yszhou4t...@gmail.com
+
+ * wget.texi: Add documentation for --start-pos.
+
2013-10-06 Tim Ruehsen tim.rueh...@gmx.de
* wget.texi: add/explain quoting of wildcard patterns
diff --git a/doc/wget.texi b/doc/wget.texi
index 4a1f7f1..87fef7c 100644
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -701,6 +701,23 @@ Another instance where you'll get a garbled file if you try to use
Note that @samp{-c} only works with @sc{ftp} servers and with @sc{http}
servers that support the @code{Range} header.
+@cindex offset
+@cindex continue retrieval
+@cindex incomplete downloads
+@cindex resume download
+@cindex start position
+@item --start-pos=@var{OFFSET}
+Start the download at position @var{OFFSET}. Offset may be expressed in bytes,
+kilobytes with the `k' suffix, or megabytes with the `m' suffix.
+
+When specified, it would override the behavior of @samp{--continue}. When
+using this option, you may also want to explicitly specify an output filename
+with @samp{-O FILE} in order to not overwrite an existing partially downloaded
+file.
+
+Server support for resuming download is needed, otherwise @samp{--start-pos}
+cannot help. See @samp{-c} for details.
+
@cindex progress indicator
@cindex dot style
@item --progress=@var{type}
diff --git a/src/ChangeLog b/src/ChangeLog
index 42ce3e4..ab8a496 100644
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,12 @@
+2013-12-21 Yousong Zhou yszhou4t...@gmail.com
+
+ * options.h: Add option --start-pos to specify start position of
+ a download.
+ * main.c: Same purpose as above.
+ * init.c: Same purpose as above.
+ * http.c: Utilize opt.start_pos for HTTP download.
+ * ftp.c: Utilize opt.start_pos for FTP retrieval.
+
2013-11-02 Giuseppe Scrivano gscri...@redhat.com
* http.c (gethttp): Increase max header value length to 512.
diff --git a/src/ftp.c b/src/ftp.c
index c2522ca..c7ab6ef 100644
--- a/src/ftp.c
+++ b/src/ftp.c
@@ -1632,6 +1632,8 @@ ftp_loop_internal (struct url *u, struct fileinfo *f, ccon *con, char **local_fi
/* Decide whether or not to restart. */
if (con-cmd DO_LIST)
restval = 0;
+ else if (opt.start_pos)
+restval = opt.start_pos;
else if (opt.always_rest
stat (locf, st) == 0
S_ISREG (st.st_mode))
diff --git a/src/http.c b/src/http.c
index 754b7ec..a354c6b 100644
--- a/src/http.c
+++ b/src/http.c
@@ -3098,6 +3098,8 @@ Spider mode enabled. Check if remote file exists.\n));
/* Decide whether or not to restart. */
if (force_full_retrieve)
hstat.restval = hstat.len;
+ else if (opt.start_pos)
+hstat.restval = opt.start_pos;
else if (opt.always_rest
got_name
stat (hstat.local_file, st) == 0
diff --git a/src/init.c b/src/init.c
index 84ae654..7f7a34e 100644
--- a/src/init.c
+++
Re: [Bug-wget] wget alpha release 1.14.96-38327
Darshit Shah dar...@gmail.com writes: @Noel: Could you please elaborate on the patches that fixes bugs in your bugtracker? @Giuseppe: Do we have any blocking bugs that need fixing? If not, would it be possible to make a release before the end of the year? If anything needs fixing, I'd like to help and ensure a new release ASAP. I've delayed it since there were some new bug reports and I had no time to go trough all of them. From a first check, it seems there is nothing blocking a release, so I will probably do that in the next few days. Is there something we should absolutely consider for inclusion before we make a new release? Giuseppe
Re: [Bug-wget] wget alpha release 1.14.96-38327
I can't think of any bug/feature request that we absolutely need to address in the next release. There is the new patch for adding a --start-pos command, but I would prefer to test it out first, before adding it as part of a new release. If anyone has any ideas / suggestions, lets please discuss them ASAP. On Sun, Dec 22, 2013 at 5:51 PM, Giuseppe Scrivano gscriv...@gnu.orgwrote: Darshit Shah dar...@gmail.com writes: @Noel: Could you please elaborate on the patches that fixes bugs in your bugtracker? @Giuseppe: Do we have any blocking bugs that need fixing? If not, would it be possible to make a release before the end of the year? If anything needs fixing, I'd like to help and ensure a new release ASAP. I've delayed it since there were some new bug reports and I had no time to go trough all of them. From a first check, it seems there is nothing blocking a release, so I will probably do that in the next few days. Is there something we should absolutely consider for inclusion before we make a new release? Giuseppe -- Thanking You, Darshit Shah
Re: [Bug-wget] wget alpha release 1.14.96-38327
Darshit Shah dar...@gmail.com writes: I can't think of any bug/feature request that we absolutely need to address in the next release. There is the new patch for adding a --start-pos command, but I would prefer to test it out first, before adding it as part of a new release. yes, it is too late to take a new feature that still has to be properly discussed. Giuseppe
Re: [Bug-wget] [PATCH] Make wget capable of starting download from a specified position.
On Sat, Dec 21, 2013 at 11:05:01AM +0100, Dagobert Michelsen wrote: Hi, Am 21.12.2013 um 10:24 schrieb Yousong Zhou yszhou4t...@gmail.com: In my situation, wget was trigger on the remote machine like the following: wget -O - --start-pos $OFFSET $URL | nc -lp 7193 Then on local machine, I would download with: nc localhost 7193 Before these, a local forwarding tunnel has been setup with ssh to make this possible. So in this case, there was no local file on the machine where wget was triggerred and `--continue' will not work. I am sure there are other cases `--start-pos' would be useful and that `--start-pos' would make wget more complete. When I just look at your problem it seems to be easier to set up the tunnel slightly different and pull with standard wget. If the URL looks like http://host:port/rest and you set up the tunnel with ssh -L 7193:host:port machine-where-you-called-wget-previously and then just wget http://localhost:7193/rest the range requests would be sent by wget just fine to the initial server and you could also safely use -c on further wget invocations (or with proper values for -t / -T automatically). Just tried this approach. It did not work out as expected because HTTP server responded with multiple levels of redirection thus host part were changed on the fly. Anyway, thank you for you time. yousong
[Bug-wget] [PATCH] sample.wgetrc: add links tot he manual
--- doc/sample.wgetrc | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/sample.wgetrc b/doc/sample.wgetrc index eaf2bc8..ad86be3 100644 --- a/doc/sample.wgetrc +++ b/doc/sample.wgetrc @@ -5,7 +5,10 @@ ## You can use this file to change the default behaviour of wget or to ## avoid having to type many many command-line options. This file does ## not contain a comprehensive list of commands -- look at the manual -## to find out what you can put into this file. +## to find out what you can put into this file. You can find this here: +## $ info wget.info 'Startup File' +## Or online here: +## https://www.gnu.org/software/wget/manual/wget.html#Startup-File ## ## Wget initialization file can reside in /usr/local/etc/wgetrc ## (global, for all users) or $HOME/.wgetrc (for a single user). -- 1.8.4.3
[Bug-wget] [PATCH v4] Make wget capable of starting download from a specified position.
This patch adds an option `--start-pos' for specifying starting position
of a download, both for HTTP and FTP. When specified, the newly added
option would override `--continue'. Apart from that, no existing code
should be affected.
Signed-off-by: Yousong Zhou yszhou4t...@gmail.com
---
v3 - v4
In doc/wget.texi and wget usage output, explicitly note that
--start-pos is zero-based.
v2 - v3
Fix a typo and add description text for the new option into the usage
output. Thank Darshit Shah dar...@gmail.com for the suggestions.
v1 - v2
It was kindly pointed out by Darshit Shah dar...@gmail.com that
server support for resuming download is required, so adding this into
doc/wget.texi.
doc/ChangeLog |4
doc/wget.texi | 17 +
src/ChangeLog |9 +
src/ftp.c |2 ++
src/http.c|2 ++
src/init.c|1 +
src/main.c|3 +++
src/options.h |1 +
8 files changed, 39 insertions(+), 0 deletions(-)
From d8fd955d161bd8ba17ac97cbcf7a3ed316e00630 Mon Sep 17 00:00:00 2001
From: Yousong Zhou yszhou4t...@gmail.com
Date: Fri, 20 Dec 2013 23:17:43 +0800
Subject: [PATCH v4] Make wget capable of starting download from a specified position.
This patch adds an option `--start-pos' for specifying starting position
of a download, both for HTTP and FTP. When specified, the newly added
option would override `--continue'. Apart from that, no existing code
should be affected.
Signed-off-by: Yousong Zhou yszhou4t...@gmail.com
---
v3 - v4
In doc/wget.texi and wget usage output, explicitly note that
--start-pos is zero-based.
v2 - v3
Fix a typo and add description text for the new option into the usage
output. Thank Darshit Shah dar...@gmail.com for the suggestions.
v1 - v2
It was kindly pointed out by Darshit Shah dar...@gmail.com that
server support for resuming download is required, so adding this into
doc/wget.texi.
doc/ChangeLog |4
doc/wget.texi | 17 +
src/ChangeLog |9 +
src/ftp.c |2 ++
src/http.c|2 ++
src/init.c|1 +
src/main.c|3 +++
src/options.h |1 +
8 files changed, 39 insertions(+), 0 deletions(-)
diff --git a/doc/ChangeLog b/doc/ChangeLog
index 3b05756..df103c8 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,7 @@
+2013-12-21 Yousong Zhou yszhou4t...@gmail.com
+
+ * wget.texi: Add documentation for --start-pos.
+
2013-10-06 Tim Ruehsen tim.rueh...@gmx.de
* wget.texi: add/explain quoting of wildcard patterns
diff --git a/doc/wget.texi b/doc/wget.texi
index 4a1f7f1..5094c26 100644
--- a/doc/wget.texi
+++ b/doc/wget.texi
@@ -701,6 +701,23 @@ Another instance where you'll get a garbled file if you try to use
Note that @samp{-c} only works with @sc{ftp} servers and with @sc{http}
servers that support the @code{Range} header.
+@cindex offset
+@cindex continue retrieval
+@cindex incomplete downloads
+@cindex resume download
+@cindex start position
+@item --start-pos=@var{OFFSET}
+Start downloading at zero-based position @var{OFFSET}. Offset may be expressed
+in bytes, kilobytes with the `k' suffix, or megabytes with the `m' suffix.
+
+When specified, it would override the behavior of @samp{--continue}. When
+using this option, you may also want to explicitly specify an output filename
+with @samp{-O FILE} in order to not overwrite an existing partially downloaded
+file.
+
+Server support for resuming download is needed, otherwise @samp{--start-pos}
+cannot help. See @samp{-c} for details.
+
@cindex progress indicator
@cindex dot style
@item --progress=@var{type}
diff --git a/src/ChangeLog b/src/ChangeLog
index 42ce3e4..ab8a496 100644
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,12 @@
+2013-12-21 Yousong Zhou yszhou4t...@gmail.com
+
+ * options.h: Add option --start-pos to specify start position of
+ a download.
+ * main.c: Same purpose as above.
+ * init.c: Same purpose as above.
+ * http.c: Utilize opt.start_pos for HTTP download.
+ * ftp.c: Utilize opt.start_pos for FTP retrieval.
+
2013-11-02 Giuseppe Scrivano gscri...@redhat.com
* http.c (gethttp): Increase max header value length to 512.
diff --git a/src/ftp.c b/src/ftp.c
index c2522ca..c7ab6ef 100644
--- a/src/ftp.c
+++ b/src/ftp.c
@@ -1632,6 +1632,8 @@ ftp_loop_internal (struct url *u, struct fileinfo *f, ccon *con, char **local_fi
/* Decide whether or not to restart. */
if (con-cmd DO_LIST)
restval = 0;
+ else if (opt.start_pos)
+restval = opt.start_pos;
else if (opt.always_rest
stat (locf, st) == 0
S_ISREG (st.st_mode))
diff --git a/src/http.c b/src/http.c
index 754b7ec..a354c6b 100644
--- a/src/http.c
+++ b/src/http.c
@@ -3098,6 +3098,8 @@ Spider mode enabled. Check if remote file exists.\n));
/* Decide whether or not to restart. */
if (force_full_retrieve)
hstat.restval =
