Re: wget 1.11 alpha1 [Fwd: Bug#378691: wget --continue doesn't workwith HTTP]
Jochen Roderburg wrote: I have now tested the new wget 1.11 beta1 on my Linux system and the above issue is solved now. The Remote file is newer message now only appears when the local file exists and most of the other logic with time-stamping and file-naming works like expected. excellent. I meanwhile found, however, another new problem with time-stamping, which mainly occurs in connection with a proxy-cache, I will report that in a new thread. Same for a small problem with the SSL configuration. thank you very much for the useful bug reports you keep sending us ;-) -- Aequam memento rebus in arduis servare mentem... Mauro Tortonesi http://www.tortonesi.com University of Ferrara - Dept. of Eng.http://www.ing.unife.it GNU Wget - HTTP/FTP file retrieval tool http://www.gnu.org/software/wget Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net Ferrara Linux User Group http://www.ferrara.linux.it
Re: wget 1.11 alpha1 [Fwd: Bug#378691: wget --continue doesn't workwith HTTP]
Jochen Roderburg ha scritto: Zitat von Jochen Roderburg [EMAIL PROTECTED]: Zitat von Hrvoje Niksic [EMAIL PROTECTED]: Mauro, you will need to look at this one. Part of the problem is that Wget decides to save to index.html.1 although -c is in use. That is solved with the patch attached below. But the other part is that hstat.local_file is a NULL pointer when stat(hstat.local_file, st) is used to determine whether the file already exists in the -c case. That seems to be a result of your changes to the code -- previously, hstat.local_file would get initialied in http_loop. This looks as if if could also be the cause for the problems which I reported some weeks ago for the timestamping mode (http://www.mail-archive.com/wget@sunsite.dk/msg09083.html) Hello Mauro, The timestamping issues I reported in above mentioned message are now also repaired by the patch you mailed last week here. Only the small *cosmetic* issue remains that it *always* says: Remote file is newer, retrieving. even if there is no local file yet. hi jochen, i have been working on the problem you reported for the last couple of days. i've just committed a patch that should fix it for good. could you please try the new HTTP code and tell me if it works properly? thank you very much for your help. -- Aequam memento rebus in arduis servare mentem... Mauro Tortonesi http://www.tortonesi.com University of Ferrara - Dept. of Eng.http://www.ing.unife.it GNU Wget - HTTP/FTP file retrieval tool http://www.gnu.org/software/wget Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net Ferrara Linux User Group http://www.ferrara.linux.it
Re: wget 1.11 alpha1 [Fwd: Bug#378691: wget --continue doesn't workwith HTTP]
Zitat von Jochen Roderburg [EMAIL PROTECTED]: Zitat von Hrvoje Niksic [EMAIL PROTECTED]: Mauro, you will need to look at this one. Part of the problem is that Wget decides to save to index.html.1 although -c is in use. That is solved with the patch attached below. But the other part is that hstat.local_file is a NULL pointer when stat(hstat.local_file, st) is used to determine whether the file already exists in the -c case. That seems to be a result of your changes to the code -- previously, hstat.local_file would get initialied in http_loop. This looks as if if could also be the cause for the problems which I reported some weeks ago for the timestamping mode (http://www.mail-archive.com/wget@sunsite.dk/msg09083.html) Hello Mauro, The timestamping issues I reported in above mentioned message are now also repaired by the patch you mailed last week here. Only the small *cosmetic* issue remains that it *always* says: Remote file is newer, retrieving. even if there is no local file yet. J.Roderburg
Re: wget 1.11 alpha1 [Fwd: Bug#378691: wget --continue doesn't workwith HTTP]
Hrvoje Niksic ha scritto:
Noèl Köthe [EMAIL PROTECTED] writes:
a wget -c problem report with the 1.11 alpha 1 version
(http://bugs.debian.org/378691):
I can reproduce the problem. If I have already 1 MB downloaded wget -c
doesn't continue. Instead it starts to download again:
Mauro, you will need to look at this one. Part of the problem is that
Wget decides to save to index.html.1 although -c is in use. That is
solved with the patch attached below. But the other part is that
hstat.local_file is a NULL pointer when
stat(hstat.local_file, st) is used to determine whether the file
already exists in the -c case. That seems to be a result of your
changes to the code -- previously, hstat.local_file would get
initialied in http_loop.
The partial patch follows:
Index: src/http.c
===
--- src/http.c (revision 2178)
+++ src/http.c (working copy)
@@ -1762,7 +1762,7 @@
return RETROK;
}
- else
+ else if (!ALLOW_CLOBBER)
{
char *unique = unique_name (hs-local_file, true);
if (unique != hs-local_file)
you're right, of course. the patch included in attachment should fix the
problem. since the new HTTP code supports Content-Disposition and delays the
decision of the destination filename until it receives the response header, the
best solution i could find to make -c work is to send a HEAD request to
determine the actual destination filename before resuming download if -c is given.
please, let me know what you think.
--
Aequam memento rebus in arduis servare mentem...
Mauro Tortonesi http://www.tortonesi.com
University of Ferrara - Dept. of Eng.http://www.ing.unife.it
GNU Wget - HTTP/FTP file retrieval tool http://www.gnu.org/software/wget
Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net
Ferrara Linux User Group http://www.ferrara.linux.it
Index: http.c
===
--- http.c (revisione 2178)
+++ http.c (copia locale)
@@ -1762,7 +1762,7 @@
return RETROK;
}
- else
+ else if (!ALLOW_CLOBBER)
{
char *unique = unique_name (hs-local_file, true);
if (unique != hs-local_file)
@@ -2231,6 +2231,7 @@
{
int count;
bool got_head = false; /* used for time-stamping */
+ bool got_name = false;
char *tms;
const char *tmrate;
uerr_t err, ret = TRYLIMEXC;
@@ -2264,7 +2265,10 @@
hstat.referer = referer;
if (opt.output_document)
+{
hstat.local_file = xstrdup (opt.output_document);
+ got_name = true;
+}
/* Reset the counter. */
count = 0;
@@ -2309,13 +2313,16 @@
/* Default document type is empty. However, if spider mode is
on or time-stamping is employed, HEAD_ONLY commands is
encoded within *dt. */
- if ((opt.spider !opt.recursive) || (opt.timestamping !got_head))
+ if ((opt.spider !opt.recursive)
+ || (opt.timestamping !got_head)
+ || (opt.always_rest !got_name))
*dt |= HEAD_ONLY;
else
*dt = ~HEAD_ONLY;
/* Decide whether or not to restart. */
if (opt.always_rest
+ got_name
stat (hstat.local_file, st) == 0
S_ISREG (st.st_mode))
/* When -c is used, continue from on-disk size. (Can't use
@@ -2484,6 +2491,12 @@
continue;
}
+ if (opt.always_rest !got_name)
+{
+ got_name = true;
+ continue;
+}
+
if ((tmr != (time_t) (-1))
(!opt.spider || opt.recursive)
((hstat.len == hstat.contlen) ||
Index: ChangeLog
===
--- ChangeLog (revisione 2178)
+++ ChangeLog (copia locale)
@@ -1,3 +1,9 @@
+2006-08-16 Mauro Tortonesi [EMAIL PROTECTED]
+
+ * http.c: Fixed bug which broke --continue feature. Now if -c is
+ given, http_loop sends a HEAD request to find out the destination
+ filename before resuming download.
+
2006-08-08 Hrvoje Niksic [EMAIL PROTECTED]
* utils.c (datetime_str): Avoid code repetition with time_str.
Re: wget 1.11 alpha1 [Fwd: Bug#378691: wget --continue doesn't workwith HTTP]
Mauro Tortonesi [EMAIL PROTECTED] writes: you're right, of course. the patch included in attachment should fix the problem. since the new HTTP code supports Content-Disposition and delays the decision of the destination filename until it receives the response header, the best solution i could find to make -c work is to send a HEAD request to determine the actual destination filename before resuming download if -c is given. please, let me know what you think. I don't like the additional HEAD request, but I can't think of a better solution.
Re: wget 1.11 alpha1 [Fwd: Bug#378691: wget --continue doesn't workwith HTTP]
Hrvoje Niksic ha scritto: Mauro Tortonesi [EMAIL PROTECTED] writes: you're right, of course. the patch included in attachment should fix the problem. since the new HTTP code supports Content-Disposition and delays the decision of the destination filename until it receives the response header, the best solution i could find to make -c work is to send a HEAD request to determine the actual destination filename before resuming download if -c is given. please, let me know what you think. I don't like the additional HEAD request, but I can't think of a better solution. same for me. in order to avoid the overhead of the extra HEAD request, i had considered disabling Content-Disposition and using url_file_name to determine the destination filename in case -c is given. but i really didn't like that solution. -- Aequam memento rebus in arduis servare mentem... Mauro Tortonesi http://www.tortonesi.com University of Ferrara - Dept. of Eng.http://www.ing.unife.it GNU Wget - HTTP/FTP file retrieval tool http://www.gnu.org/software/wget Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net Ferrara Linux User Group http://www.ferrara.linux.it
Re: wget 1.11 alpha1 [Fwd: Bug#378691: wget --continue doesn't workwith HTTP]
Hrvoje Niksic wrote: Noèl Köthe [EMAIL PROTECTED] writes: a wget -c problem report with the 1.11 alpha 1 version (http://bugs.debian.org/378691): I can reproduce the problem. If I have already 1 MB downloaded wget -c doesn't continue. Instead it starts to download again: Mauro, you will need to look at this one. i surely will. unfortunately, at the moment i am attending the winsys 2006 research conference: http://www.winsys.org i'll take a look at the problem as soon as i get back to italy. -- Aequam memento rebus in arduis servare mentem... Mauro Tortonesi http://www.tortonesi.com University of Ferrara - Dept. of Eng.http://www.ing.unife.it GNU Wget - HTTP/FTP file retrieval tool http://www.gnu.org/software/wget Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net Ferrara Linux User Group http://www.ferrara.linux.it
Re: wget 1.11 alpha1 [Fwd: Bug#378691: wget --continue doesn't workwith HTTP]
Zitat von Hrvoje Niksic [EMAIL PROTECTED]: Mauro, you will need to look at this one. Part of the problem is that Wget decides to save to index.html.1 although -c is in use. That is solved with the patch attached below. But the other part is that hstat.local_file is a NULL pointer when stat(hstat.local_file, st) is used to determine whether the file already exists in the -c case. That seems to be a result of your changes to the code -- previously, hstat.local_file would get initialied in http_loop. This looks as if if could also be the cause for the problems which I reported some weeks ago for the timestamping mode (http://www.mail-archive.com/wget@sunsite.dk/msg09083.html) J.Roderburg
