Re: Problem with wget 1.11. Please help

[Micah Cowan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Amit Patel wrote:
> Ya. I have checked it properly. It checks certificates. If i don't
> specify /etc/ca-bundle.crt with my working version of 1.10.2, it
> provides "Self-signed certificate encountered" error and fails.

Er, I'm not sure, but I think I would expect that to be a problem,
regardless of whether /etc/ca-bundle.crt.

Could you please verify whether a stock wget-1.10.2, configured and
installed from our source tarballs (not RedHat's), available at
ftp.gnu.org/gnu/wget/wget-1.10.2.tar.gz ?

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer,
and GNU Wget Project Maintainer.
http://micah.cowan.name/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFH9isf7M8hyUobTrERApgfAJ4801HnL9/5W4QdcLMF87KNnutHJgCeP2bm
8rU8dsRxa4s76i/a3pnaeZ4=
=MLQZ
-END PGP SIGNATURE-

Re: Problem with wget 1.11. Please help

[Micah Cowan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Amit Patel wrote:
> Hi  Hrvoje Niksic / Micah Cowan
> 
> I am having strange problem with wget 1.11 version. While accessing a
> webserver which is running on Verisign issued Trial certificates, wget
> 1.11 gives following error even though that certificate is valid.
> 
> [EMAIL PROTECTED]:/root $ wget --ca-certificate=/etc/ca-bundle.crt
> https://rw01246.einfochips.com*



> However , if i try same with wget version 1.10.2  which is on another
> redhat machine, it is working perfect. I am not able to understand where
> the problem is. If you can spare some time and  help  help me to solve
> the issue , that would be great.

RedHat has been known to modify Wget fairly heavily. Recent versions of
RedHat's "Wget 1.10.2" differ very substantially from ours. It would be
more useful to see how a wget-1.10.2 that was built straight from our
source packages would compare.

Is /etc/ca-bundle.crt exactly the same on both systems?

And: are you sure the "working" version doesn't have "check_certificate
= off" in either the /etc/wgetrc or ~/.wgetrc?

...If you can verify all of that, then I'm not sure how to go forward:
we'd need access to try that server, I guess (which doesn't resolve, for
me).

Also: Hrvoje and I are both subscribed to this list; no need to Cc us.

It's recommended, when posting to lists, that you choose a subject
header that describes the problem you're having, rather than just
describing that you have a problem. "Valid SSL certificate not accepted"
would have been a reasonable subject.

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer,
and GNU Wget Project Maintainer.
http://micah.cowan.name/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFH9RET7M8hyUobTrERArgYAJ9MwEOct/mhd0ic9/HzMTvuLXV74ACeJpvK
Ub3NwgJB5fLe4Z11agbCOWk=
=J4Pg
-END PGP SIGNATURE-

Problem with wget 1.11. Please help

[Amit Patel]

Hi  Hrvoje Niksic / Micah Cowan

I am having strange problem with wget 1.11 version. While accessing a 
webserver which is running on Verisign issued Trial certificates, wget 
1.11 gives following error even though that certificate is valid.


[EMAIL PROTECTED]:/root $ wget --ca-certificate=/etc/ca-bundle.crt 
https://rw01246.einfochips.com*


--2008-04-04 21:22:57--  https://rw01246.einfochips.com/
Resolving rw01246.einfochips.com... 10.101.1.246
Connecting to rw01246.einfochips.com|10.101.1.246|:443... connected.
ERROR: cannot verify rw01246.einfochips.com's certificate, issued by 
`/C=US/O=VeriSign, Inc./OU=For Test Purposes Only.  No 
assurances./OU=Terms of use at https://www.verisign.com/cps/testca 
(c)05/CN=VeriSign Trial Secure Server Test CA':

/*  certificate signature failure*/
To connect to rw01246.einfochips.com insecurely, use 
`--no-check-certificate'.

Unable to establish SSL connection.


However , if i try same with wget version 1.10.2  which is on another 
redhat machine, it is working perfect. I am not able to understand where 
the problem is. If you can spare some time and  help  help me to solve 
the issue , that would be great.


Following is the output of "ldd" command on both the machines. This may 
help you in debugging the issue.


*For wget version 1.11 (This version is having problem.)*

   libdl.so.2 => /lib/libdl.so.2 (0x40019000) [actual file - 
libdl-2.2.4.so ]
   librt.so.1 => /lib/librt.so.1 (0x4001d000) [actual file - 
librt-2.2.4.so ]
   libssl.so.0 => /usr/lib/libssl.so.0 (0x4003) [actual file - 
libssl.so.0.9.5a ]
   libcrypto.so.0 => /usr/lib/libcrypto.so.0 (0x4005e000) [actual 
file - libcrypto.so.0.9.5a ]
   libc.so.6 => /lib/libc.so.6 (0x4011a000) [actual file - 
libc-2.2.4.so ]
   libpthread.so.0 => /lib/libpthread.so.0 (0x40233000) [actual 
file -  libpthread-0.9.so ]
   /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x4000) [actual 
file - ld-2.2.4.so ]


*For wget version 1.10.2 (This works fine.)*  


   linux-gate.so.1 =>  (0x0064c000)
   libssl.so.6 => /lib/libssl.so.6 (0x0045b000) [actual file - 
libssl.so.0.9.8b]
   libcrypto.so.6 => /lib/libcrypto.so.6 (0x001dd000) [actual file 
- libcrypto.so.0.9.8b]
   libdl.so.2 => /lib/libdl.so.2 (0x00c4) [actual file - 
libdl-2.5.so]
   libz.so.1 => /usr/lib/libz.so.1 (0x00c88000) [actual file - 
libz.so.1.2.3]
   librt.so.1 => /lib/i686/nosegneg/librt.so.1 (0x060f9000) [actual 
file - librt-2.5.so]
   libc.so.6 => /lib/i686/nosegneg/libc.so.6 (0x00afd000) [actual 
file - libc-2.5.so]
   libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0x00373000) 
[actual file - libgssapi_krb5.so.2.2]
   libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x003a) [actual file 
- libkrb5.so.3.2]
   libcom_err.so.2 => /lib/libcom_err.so.2 (0x0033b000) [actual 
file - libcom_err.so.2.1]
   libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x00433000) 
[actual file - libk5crypto.so.3.0]
   libresolv.so.2 => /lib/libresolv.so.2 (0x00326000) [actual file 
- libresolv-2.5.so]

   /lib/ld-linux.so.2 (0x0012e000)
   libpthread.so.0 => /lib/i686/nosegneg/libpthread.so.0 
(0x00c6f000) [actual file - libpthread-2.5.so]
   libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0x00429000) 
[actual file - libkrb5support.so.0.1]



Waiting for positive response.

Thanks in advance,
Amit Patel
--
_
Disclaimer: This e-mail message and all attachments transmitted with it
are intended solely for the use of the addressee and may contain legally
privileged and confidential information. If the reader of this message
is not the intended recipient, or an employee or agent responsible for
delivering this message to the intended recipient, you are hereby
notified that any dissemination, distribution, copying, or other use of
this message or its attachments is strictly prohibited. If you have
received this message in error, please notify the sender immediately by
replying to this message and please delete it from your computer. Any
views expressed in this message are those of the individual sender
unless otherwise stated.Company has taken enough precautions to prevent
the spread of viruses. However the company accepts no liability for any
damage caused by any virus transmitted by this email.
__

Re: Problem with wget (v1.11) and FTP URL including ..

[Micah Cowan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Micah Cowan wrote:

> it looks to me like your URL
> (ftp://ftp.xxx.pwp.blueyonder.co.uk/../logs/access.20080309) is
> being transmuted by some URL "cleanup" code well before it ever gets to
> the FTP handle. My guess is that this change was intentionally done, as
> such a URL for HTTP would probably be wrong (I'm not sure that it's
> actually ill-formed; but for safety reasons it was probably a good idea
> to remove ".." from the beginning of paths in HTTP). However, it's
> perfectly fine for FTP URLs, and removing it for them is misbehavior.

This change was introduced in 2006 Feb, at the suggestion of Frank
McCown: http://article.gmane.org/gmane.comp.web.wget.general/5290/

The change can be seen here:
http://hg.addictivecode.org/wget/mainline/rev/798506c1ce67/

The change was introduced to comply with the recommended algorithm for
normalizing paths in RFC 3986 (section 5.2.4). So the change was
deliberate, and done to comply with recommended practice.

The thing is, of course, is that it sort of breaks the FTP scheme
defined in RFC 1738. Are there any updates to the FTP URL definition? I
don't see any (it seems to me that the RFC index would show an
"Updated-By" for 1738 to FTP URL updates, as it does for gopher and mailto).

I'm going to punt this so it doesn't hold up 1.11.1 (speaking of which,
has anyone been banging on the prerelease? :\ ). I vote that we ignore
what 3986 says with respect to ftp paths, perhaps passing a flag to the
path_simplify function so it knows whether to do that or not?

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer...
http://micah.cowan.name/

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFH4aoa7M8hyUobTrERAnh2AJ9TBQwyknhPl9pUFzVuTnk9LtQ0rQCcCTO0
VNz+Bv6tVv6KvSl19yIVeBs=
=nRHg
-END PGP SIGNATURE-

Re: Problem with wget (v1.11) and FTP URL including ..

[Micah Cowan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Richard wrote:
> Hi Micah,
> 
> I've tried to post this reply to the mailing list but it's getting
> blocked by SpamAssassin so having to reply directly - hope that's OK.

Sure; it's probably due to the "xxx" strings within URLs. :)

I was thinking perhaps the "working" case with Wget 1.11 would be using
a so-called "FTP proxy"; an FTP server that proxies to other servers. Of
course, the FTP-handling logic for FTP-over-HTTP proxies is entirely
handled by the HTTP proxy server, so it makes sense that it would work
there.

The logs look like enough to fish out the problem, so I'll do some
snooping around to see what can be done about this.

Judging from this bit from the failing Wget 1.11 logs:

- --2008-03-10 18:59:10--
ftp://ftp.xxx.pwp.blueyonder.co.uk/logs/access.20080309.gz
Host `ftp.xxx.pwp.blueyonder.co.uk' has not issued a general basic
challenge.
Resolving webcache.virginmedia.com... 195.188.152.6

it looks to me like your URL
(ftp://ftp.xxx.pwp.blueyonder.co.uk/../logs/access.20080309) is
being transmuted by some URL "cleanup" code well before it ever gets to
the FTP handle. My guess is that this change was intentionally done, as
such a URL for HTTP would probably be wrong (I'm not sure that it's
actually ill-formed; but for safety reasons it was probably a good idea
to remove ".." from the beginning of paths in HTTP). However, it's
perfectly fine for FTP URLs, and removing it for them is misbehavior.

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer...
http://micah.cowan.name/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFH1Y8t7M8hyUobTrERAietAJ4pzcZ9vVOTk4Bsy9wn89J5oHCyTQCdEgbR
VtaAtKiIhh1lXmTi9CAB9do=
=vfM1
-END PGP SIGNATURE-

Re: Problem with wget (v1.11) and FTP URL including ..

[Micah Cowan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Richard wrote:
> Hi,
> 
> Since upgrading wget from 1.10.2 to 1.11 (on a Sun Solaris 9 server) I
> am no longer able to retrieve files with a command similar to:
> 
> wget --user=xxx --password=xxx --output-document=logfile.txt
> ftp://ftp.username.myby.co.uk/../logs/logfile.txt
> 
> It reports the following:
> 
> ==> SYST ... done.==> PWD ... done.
> ==> TYPE I ... done.  ==> CWD /htdocs/logs ...
> No such directory `logs'.
> 
> The directory htdocs is the 'home' directory, and so it looks as if wget
> has not gone up one directory (ie. done the */../* part of the URL),
> before going into logs.
> 
> This worked fine in 1.10.2, and I have also just discovered that it DOES
> work in 1.11 if I go via a proxy server!
> 
> *Please CC me in on any replies as I have not subscribed to this list.*

I'd need to see the full logs (with --debug set) for both Wget 1.10.2
and the working (via proxy) and not-working Wget 1.11 cases. Or else, an
example URL that we can test directly that gives this behavior.

I'll try to set up a similar test of my own when I have a chance, but
the fact that it works when using a proxy makes me think that it's
server-dependent behavior, so I'll probably still end up needing logs.

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer...
http://micah.cowan.name/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD4DBQFH1XvD7M8hyUobTrERAuc7AKCHsMrHIkazzhNkAJtrG0epumGewwCXbp5G
c+DE7xGxTu1I8kdWJk5XzA==
=a1Gm
-END PGP SIGNATURE-

Problem with wget (v1.11) and FTP URL including ..

[Richard]

Hi,

Since upgrading wget from 1.10.2 to 1.11 (on a Sun Solaris 9 server) I am no
longer able to retrieve files with a command similar to:

wget --user=xxx --password=xxx --output-document=logfile.txt
ftp://ftp.username.myby.co.uk/../logs/logfile.txt

It reports the following:

==> SYST ... done.==> PWD ... done.
==> TYPE I ... done.  ==> CWD /htdocs/logs ...
No such directory `logs'.

The directory htdocs is the 'home' directory, and so it looks as if wget has
not gone up one directory (ie. done the */../* part of the URL), before
going into logs.

This worked fine in 1.10.2, and I have also just discovered that it DOES
work in 1.11 if I go via a proxy server!

*Please CC me in on any replies as I have not subscribed to this list.*

Thanks,

Richard van der Leeden

Problem with Wget on windows

[Hunny Garg]

Hi
I am trying to download from a ftp server using wget on windows. The
problem I am facing is that if there a directory having space in its
name then wget replaces that space character with @20 after downloading
on my machine. The client and server are both on windows machine
Basically I  want to maintain exactly the same directory structure as it
is on server machineis there anyway to achieve that.

Thanx in advance

Re: Problem with WGET

[Micah Cowan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Sorokin Nikita wrote:
> Hello!
> I want to download all files with .RAR extension in specified directory
> (http://test.com/test/ and all RAR files...), but I don't know how to
> dothat.
> Please, help me...

Generally, real, live URLs are preferred to fake ones, to help us help you.

Have you tried --accept=.RAR,.rar ?

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer...
http://micah.cowan.name/

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHQeJu7M8hyUobTrERCGobAJ4jHWJK0k0NICH/46qCCD57yzU9PACfTvbj
FoqVxkHvFFCUcf/CevZZ4ZI=
=rAss
-END PGP SIGNATURE-

Problem with WGET

[Sorokin Nikita]

Hello!
I want to download all files with .RAR extension in specified directory
(http://test.com/test/ and all RAR files...), but I don't know how to dothat.
Please, help me...

Thanks,
N.S.

Cookie problem with WGet after 1.9.1...

[Mike Peck]

I've narrowed down what I think is a bug in cookie handling related to http 
authorization digests in the last three versions of wget.  I found that 1.9.1 
does the right thing, but that all later versions do not.

What I am doing, is a request like this:

wget -O usb.tivo --http-user=tivo --http-passwd=2807747290 
"http://192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960";


What I get at the console that issued this command is (using 1.10.2):

 /usr/local/src/wget-1.10.2/src/wget -O usb.tivo --http-user=tivo 
--http-passwd=2807747290 
"http://192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960";
--07:41:48--  
http://1192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960
   => `usb.tivo'
Connecting to 192.168.23.33:80... connected.
HTTP request sent, awaiting response... 401 Digest Authorization Required
Reusing existing connection to 192.168.23.33:80.
HTTP request sent, awaiting response... 400 Bad Request
07:41:48 ERROR 400: Bad Request.






















[EMAIL PROTECTED]:~/ttcp> /usr/local/src/wget-1.10.2/src/wget -O usb.tivo 
--http-user=tivo --http-passwd=2807747290 
"http://10.200.2.196:80/download/Saved%20by%20the%20Bell.TiVo?Container=%2FNowPlaying&id=15960";
--11:01:09--  
http://10.200.2.196/download/Saved%20by%20the%20Bell.TiVo?Container=%2FNowPlaying&id=15960
   => `usb.tivo'
Connecting to 10.200.2.196:80... connected.
HTTP request sent, awaiting response... 401 Digest Authorization Required
Reusing existing connection to 10.200.2.196:80.
HTTP request sent, awaiting response... 400 Bad Request
11:01:10 ERROR 400: Bad Request.

[EMAIL PROTECTED]:~/ttcp> scp [EMAIL PROTECTED]:/sandbox/archive/wget-1.8* .
[EMAIL PROTECTED]'s password: 
wget-1.8.1.tar.gz 100% 1072KB   1.1MB/s   00:00
[EMAIL PROTECTED]:~/ttcp> /usr/local/src/wget-1.8.1/src/wget -O usb.tivo 
--http-user=tivo --http-passwd=2807747290 
"http://10.200.2.196:80/download/Saved%20by%20the%20Bell.TiVo?Container=%2FNowPlaying&id=15960";
--11:13:51--  
http://10.200.2.196/download/Saved%20by%20the%20Bell.TiVo?Container=%2FNowPlaying&id=15960
   => `usb.tivo'
Connecting to 10.200.2.196:80... connected.
HTTP request sent, awaiting response... 401 Digest Authorization Required
Connecting to 10.200.2.196:80... connected.
HTTP request sent, awaiting response... 200 File Follows
Length: unspecified [video/x-tivo-mpeg]

Cookie problem with WGet after 1.9.1...

[Mike Peck]

I've narrowed down what I think is a bug in cookie handling related to http 
authorization digests in the last three versions of wget.  I found that 1.9.1 
and earlier does the right thing, but that all later versions do not.

What I am doing, is a request like this:

wget -O usb.tivo --http-user=tivo --http-passwd=2807747290 
"http://192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960";


What I get at the console that issued this command is (using 1.10.2):

 /usr/local/src/wget-1.10.2/src/wget -O usb.tivo --http-user=tivo 
--http-passwd=2807747290 
"http://192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960";
--07:41:48--  
http://1192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960
   => `usb.tivo'
Connecting to 192.168.23.33:80... connected.
HTTP request sent, awaiting response... 401 Digest Authorization Required
Reusing existing connection to 192.168.23.33:80.
HTTP request sent, awaiting response... 400 Bad Request
07:41:48 ERROR 400: Bad Request.


Here's what 1.9.1 does:

 /usr/local/src/wget-1.9.1/src/wget -O usb.tivo --http-user=tivo 
--http-passwd=2807747290 
"http://192.168.23.33:80/download/foo.TiVo?Container=%2FNowPlaying&id=15960";
--07:44:29--  
http://192.168.23.33/download/foo.TiVo?Container=%2FNowPlaying&id=15960
   => `usb.tivo'
Connecting to 192.168.23.33:80... connected.
HTTP request sent, awaiting response... 401 Digest Authorization Required
Connecting to 192.168.23.33:80... connected.
HTTP request sent, awaiting response... 200 File Follows
Length: unspecified [video/x-tivo-mpeg]

[  <=>] 1,711,782799.11K/s 


So, 1.9.1 works and 1.10.2 does not.  That doesn't mean wget has a problem, it 
could be that my server is taking advantage of a bug in 1.9.1, however, looking 
at the server logs, I see the problem.

The problem is that 1.10 and later are not sending the cookies back up, even if 
I add --enable-digest at build time and --keep-session-cookies on the cmd line. 
 (The cookie is "sid" that gets ignored in 1.10.2.)


Server logs for 1.9.1:

Oct 21 14:48:25 10 [TvHttp:80:51][5477]: REQUEST: 212 bytes: GET 
/download/foo.TiVo?Container=%2FNowPlaying&id=15960 HTTP/1.0^M User-Agent: 
Wget/1.9.1^M Host: 192.168.23.33^M Accept: */*^M Connection: Keep-Alive^M 
Authorization: Basic dGl2bzoyODA3NzQ3Mjkw^M ^M 
Oct 21 14:48:25 10 [TvHttp:80:51][5477]: REPLY: 371 bytes, HTTP/1.1 401 Digest 
Authorization Required^M Server: tivo-httpd-1:b-7-2/2005.10.10-0007:6F9:alpha^M 
Set-Cookie: sid=7DB0C3425AAA0253; path=/; expires="Saturday, 16-Feb-2013 
00:00:00 GMT";^M WWW-Authenticate: Digest realm="TiVo DVR", peg: A/V Filter is 
off
nonce="3F2C4DAAA0EF594D", qop="auth"^M Content-Length: 38^M Content-Type: 
text/html^M Connection: keep-alive^M Keep-Alive: max=10, timeout=30^M ^M 
Oct 21 14:48:25 10 [TvHttp:80:52][5477]: REQUEST: 406 bytes: GET 
/download/foo.TiVo?Container=%2FNowPlaying&id=15960 HTTP/1.0^M User-Agent: 
Wget/1.9.1^M Host: 192.168.23.33^M Accept: */*^M Connection: Keep-Alive^M 
Cookie: sid=7DB0C3425AAA0253^M Authorization: Digest username="tivo", 
realm="TiVo DVR", nonce="3F2C4DAAA0EF594D", 
uri="/download/foo.TiVo?Container=%2FNowPlaying&id=15960", 
response="326cd21ece1639b3f273099803b29382"^M ^M 
Oct 21 14:48:25 10 TvHttpDownloadModule[5477]: download sid=7DB0C34257FC0253, 
rid=15960, off=0



Server logs for 1.10.2:

Oct 21 14:49:31 10 [TvHttp:80:53][5477]: REQUEST: 213 bytes: GET 
/download/foo.TiVo?Container=%2FNowPlaying&id=15960 HTTP/1.0^M User-Agent: 
Wget/1.10.2^M Accept: */*^M Authorization: Basic dGl2bzoyODA3NzQ3Mjkw^M Host: 
192.168.23.33^M Connection: Keep-Alive^M ^M 
Oct 21 14:49:31 10 [TvHttp:80:53][5477]: REPLY: 371 bytes, HTTP/1.1 401 Digest 
Authorization Required^M Server: tivo-httpd-1:b-7-2/2005.10.10-0007:6F9:alpha^M 
Set-Cookie: sid=6D3223D6865F2836; path=/; expires="Saturday, 16-Feb-2013 
00:00:00 GMT";^M WWW-Authenticate: Digest realm="TiVo DVR", 
nonce="9749DE695AAAE83A", qop="auth"^M Content-Length: 38^M Content-Type: 
text/html^M Connection: keep-alive^M Keep-Alive: max=10, timeout=30^M ^M 
Oct 21 14:49:31 10 [TvHttp:80:53][5477]: REQUEST: 377 bytes: GET 
/download/foo.TiVo?Container=%2FNowPlaying&id=15960 HTTP/1.0^M User-Agent: 
Wget/1.10.2^M Accept: */*^M Authorization: Digest username="tivo", realm="TiVo 
DVR", nonce="9749DE695AAAE83A", 
uri="/download/foo.TiVo?Container=%2FNowPlaying&id=15960", 
response="56bdebe5207862c929bc79e11816caa0"^M Host: 192.168.23.33^M Connection: 
Keep-Alive^M ^M 
Oct 21 14:49:31 10 [TvHttp:80:53][5477]: REPLY: 244 bytes, HTTP/1.1 400 Bad 
Request^M Server: tivo-httpd-1:b-7-2/2005.10.10-0007:6F9:alpha^M Set-Cookie: 
sid=6D3223D6865F2836; path=/; expires="Saturday, 16-Feb-2013 00:00:00 GMT";^M 
Content-Length: 39^M TiVo-Message: session id missing^M Connection: close^M ^M 



It looks like the change that caused the problem was making session cookies not 
used by default, and that the flag to re-enable them doesn't work.

Here's the c

Problem with wget - Error No: -1 No data received

[Anitha Srinivasan]

I have a serious problem with this wget utility.
When used in a cronscript to execute a php script say the url is like this(http://111.222.333.444/somepath/some_file_name.php), most of the timesthe output received is 
Error No: -i, No data received.

I tot theproblem mite be with the script such that the script gets timed out. Butif thts the case all the time for the same query/request it shud takethe same time, instead of getting executed sucessfully randomly.

Plsexplain me wht mite be the problem.
Thnx in advance for anyhelp.

Regds
Anitha S.

Re: Problem with wget -r -O -

[Mauro Tortonesi]

Alle 13:35, mercoledì 17 novembre 2004, Stephan Knuth ha scritto:
> Hello,

hi stephan,

first of all thank you very much for your bugreport.

> I'm not completely sure, if it's a bug or a feature I found, but doing
> something like
>
> wget --recursive --output-document=- web-document.hmtl
>
> should print out web-document.html and documents which are linked from
> web-document.html, but instead it only gets the first document.
>
> When not using "--output-document=-", I get everythink I want, but
> using standard output, it doesn't work.

yes, unfortunately that's a bug. i am opening a ticket for this in our bug 
tracking system.

> Wget complains about a file it cannot find, "web-document.html". Of
> course it cannot be found on my machine, it's only given to standard
> output.
>
> I'm using GNU Wget 1.9.1.

in the last cvs version of wget we have fixed this problem, but it seems that 
there is still some work to do in order to fix -O when downloading multiple 
files.

-- 
Aequam memento rebus in arduis servare mentem...

Mauro Tortonesi

University of Ferrara - Dept. of Eng.http://www.ing.unife.it
Institute of Human & Machine Cognition   http://www.ihmc.us
Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net
Ferrara Linux User Group http://www.ferrara.linux.it

Problem with wget -r -O -

[Stephan Knuth]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

I'm not completely sure, if it's a bug or a feature I found, but doing
something like

wget --recursive --output-document=- web-document.hmtl

should print out web-document.html and documents which are linked from
web-document.html, but instead it only gets the first document.

When not using "--output-document=-", I get everythink I want, but
using standard output, it doesn't work.

Wget complains about a file it cannot find, "web-document.html". Of
course it cannot be found on my machine, it's only given to standard
output.

I'm using GNU Wget 1.9.1.

Stephan

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQFBm0Wd76z5ZJB7UuIRAlLoAJ461vNWZksC0pDnjYTiaO5RNcFNkACfdkbc
ZNSZXXiRAZhUTOYBJwo/RZA=
=KD6u
-END PGP SIGNATURE-

Another problem with "wget -N"

[Keith Thompson]

On 2004-07-23, I reported a couple of problems with the "wget -N"
command.





I've just run into what is probably a closely related problem.

I'm using wget 1.9.1 under SPARC/SunOS 5.8.

The URL I'm having problems with is


If I try
wget -N 
it works.  If I try
wget -N -O  
it fails *if* the specified filename is the same as the actual name
of the downloaded file; otherwise, it succeeds.

In each case, the file doesn't exist before the "wget" command.

See  for a script and log file
that exhibit the problem.

-- 
Keith Thompson, San Diego Supercomputer Center
[EMAIL PROTECTED]    858-822-0853
We must do something.  This is something.  Therefore, we must do this.

A problem with Wget

[G.MURALIKRISHNA]

Hi,
I am working with a Windows XP machine.
I tried to use the Wget 1.9.1.
But I am unable to download the file.
I was behind a proxy server+firewall. Is that restricting me OR any problem?
The commands I used are
1. Wget yahoo.com
2. wget --proxy=on http_proxy=.:80/ http://rediff.com
There are many other ways I tried, But I could not do the download(in fact I tried all the exmples given, but I failed)
I am attaching the logfile.
Can you please help in this?
 
Thanks
Murali Krishna
 
		Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!--15:58:52--  ftp://http_proxy=http//192.168.214.6:80
   => `192.168.214.6%3A80'
Resolving http_proxy=http... failed: Host not found.
--15:58:55--  http://rediff.com/
   => `index.html'
Resolving rediff.com... 202.54.124.171
Connecting to rediff.com[202.54.124.171]:80... failed: No such file or directory.
Retrying.

--15:59:16--  http://rediff.com/
  (try: 2) => `index.html'
Connecting to rediff.com[202.54.124.171]:80... failed: No such file or directory.
Retrying.

--15:59:37--  http://rediff.com/
  (try: 3) => `index.html'
Connecting to rediff.com[202.54.124.171]:80... failed: No such file or directory.
Retrying.

--15:59:58--  http://rediff.com/
  (try: 4) => `index.html'
Connecting to rediff.com[202.54.124.171]:80... failed: No such file or directory.
Retrying.

--16:00:19--  http://rediff.com/
  (try: 5) => `index.html'
Connecting to rediff.com[202.54.124.171]:80... failed: No such file or directory.
Retrying.

--16:00:40--  http://rediff.com/
  (try: 6) => `index.html'
Connecting to rediff.com[202.54.124.171]:80... failed: No such file or directory.
Retrying.

--16:00:51--  http://rediff.com/
  (try: 7) => `index.html'

Re: Problem with wget password parsing

[Hrvoje Niksic]

Joerg Petersohn <[EMAIL PROTECTED]> writes:

> i had a problem with a password containing a '?' character. Wget
> reports a bad port number because the credentials passing routine
> searches for any character [@/?#;].

You can specify %3f instead of the ? character.

Problem with wget password parsing

[Joerg Petersohn]

Hi,
i had a problem with a password containing a '?' character. Wget reports 
a bad port number because the credentials passing routine searches for 
any character [@/?#;].

You may decide if this is a bug or not, but i need to change this 
parsing, because i cannot change the password on the remote site.

Example:
-
wget -mr 'ftp://user:[EMAIL PROTECTED]/dir'
ftp://user:[EMAIL PROTECTED]/dir: Bad port number.
Code location:
File: wget-1.9.1/src/url.c,
in function: url_skip_credentials()
line 471: const char *p = (const char *)strpbrk (url, "@/?#;");
 problem ---^
Please direct any answers for me to [EMAIL PROTECTED], because i 
am not subscribed to this mailing list.

Thanks
Joerg Petersohn

RE: Problem with wget 1.9 and question mark at least on windows

[Herold Heiko]

Also note, I didn't yet compile and publish the msvc windows binary for 1.9
- I suppose that was one of the beta binaries.
Heiko

-- 
-- PREVINET S.p.A. www.previnet.it
-- Heiko Herold [EMAIL PROTECTED]
-- +39-041-5907073 ph
-- +39-041-5907472 fax

> -Original Message-
> From: Hrvoje Niksic [mailto:[EMAIL PROTECTED]
> Sent: Thursday, October 23, 2003 12:12 PM
> To: Boris New
> Cc: [EMAIL PROTECTED]
> Subject: Re: Problem with wget 1.9 and question mark at least 
> on windows
> 
> 
> Sorry about that, Wget currently applies -R and -A only to file names,
> not to the query part of the URL.  Therefore there is currently no
> built-in way to do what you want.
> 
> I do plan to fix this, but Wget 1.9 was too late in the works to add
> such a feature.
> 
> The current behavior is due to many people using -R to restrict based
> on file names and file name extensions; this usage might break if -R
> also matched the query portion of the URL by default.
> 

Re: Problem with wget 1.9 and question mark at least on windows

[Hrvoje Niksic]

Sorry about that, Wget currently applies -R and -A only to file names,
not to the query part of the URL.  Therefore there is currently no
built-in way to do what you want.

I do plan to fix this, but Wget 1.9 was too late in the works to add
such a feature.

The current behavior is due to many people using -R to restrict based
on file names and file name extensions; this usage might break if -R
also matched the query portion of the URL by default.

Problem with wget 1.9 and question mark at least on windows

[Boris New]

Hi,

I tried wget 1.9 for windows from Heiko Herold 
(http://xoomer.virgilio.it/hherold/) and the problem with the filters 
and the question marks remains:
On the following page:
http://www.wordtheque.com/owa-wt/new_wordtheque.wcom_literature.literaturea_page?lang=FR&letter=A&source=search&page=1
If I want to download all the webpages containing "FR" or "fr" (after 
"?"), it's impossible.
But it's possible to download all webpages containing "page" (before "?").
I tried all the new --restrict-file-names options and that does'nt 
change anything.

Is it due to windows version? Is there a way to correct this behavior?

Thanks in advance,

Boris
http://www.lexique.org
http://www.borisnew.org
_
Envie de discuter en "live" avec vos amis ? Télécharger MSN Messenger
http://www.ifrance.com/_reloc/m la 1ère messagerie instantanée de France

possible problem with wget 1.8.2

[george goffe]

Howdy,

I'm issuing a wget from a bourn shell script and the
uri part (i.e., web.server.wtf.com/UPPERCASEDIRECTORY)
is being translated to lower case and being presented
to the server as
web.server.wtf.com/uppercasedirectory.

I don't see anything in the man pages relating to this
so either I'm doing something stupid or there's a bug
in wget (maybe). Could you check this out and let me
know please?

Regards and THANKS for the COOL code!

George...
 

=
_/_/_/_/ _/_/_/_/ _/_/_/_/ _/_/_/_/ _/_/_/_/ _/_/_/_/ -
   _/   _/   _/_/ _/_/ _/   _/
  _/  _/_/ _/_/_/_/ _/_/ _/_/_/_/ _/  _/_/ _/_/_/_/ -
 _/_/ _/   _/_/ _/_/ _/_/ _/
_/_/_/_/ _/_/_/_/ _/_/_/_/ _/_/ _/_/_/_/ _/_/_/_/ -

"It's not what you know that hurts you, It's what you know that ain't so." Wil Rogers

__
Do You Yahoo!?
Yahoo! Finance - Get real-time stock quotes
http://finance.yahoo.com

Re: Weird 302 problem with wget 1.7

[Hrvoje Niksic]

John Levon <[EMAIL PROTECTED]> writes:

> Thanks very much (wouldn't it be good to refer to the clause in the
> RFC in the comments ?)

Uh, I suppose so.  But it doesn't matter that much -- someone looking
for it will find it anyway.  Besides, it's not clear which RFC Wget
conforms to.  Web standards are messy.

Re: Weird 302 problem with wget 1.7

[John Levon]

On Mon, Jan 14, 2002 at 04:16:24PM +0100, Hrvoje Niksic wrote:

> Ok, how about this patch:
> 
> 2002-01-14  Hrvoje Niksic  <[EMAIL PROTECTED]>
> 
>   * headers.c (header_get): Strip trailing whitespace from the
>   header.

I've tested this patch, works great.

Thanks very much (wouldn't it be good to refer to the clause in the RFC
in the comments ?)

regards
john

-- 
"Now why did you have to go and mess up the child's head, so you can get another gold 
waterbed ?
 You fake-hair contact-wearing liposuction carnival exhibit, listen to my rhyme ..."

Re: Weird 302 problem with wget 1.7

[Hrvoje Niksic]

John Levon <[EMAIL PROTECTED]> writes:

> The field-content does not include any leading or trailing LWS:
>linear white space occurring before the first non-whitespace
>character of the field-value or after the last non-whitespace
>character of the field-value. Such leading or trailing LWS MAY be
>removed without changing the semantics of the field value. Any LWS
>that occurs between field-content MAY be replaced with a single SP
>before interpreting the field value or forwarding the message
>downstream.

Ok, how about this patch:

2002-01-14  Hrvoje Niksic  <[EMAIL PROTECTED]>

* headers.c (header_get): Strip trailing whitespace from the
header.

Index: src/headers.c
===
RCS file: /pack/anoncvs/wget/src/headers.c,v
retrieving revision 1.6
diff -u -r1.6 headers.c
--- src/headers.c   2001/11/16 19:57:43 1.6
+++ src/headers.c   2002/01/14 15:13:31
@@ -64,8 +64,8 @@
as much memory as necessary for it to fit.  It need not contain a
`:', thus you can use it to retrieve, say, HTTP status line.
 
-   The trailing CRLF or LF are stripped from the header, and it is
-   zero-terminated.    Is this well-behaved?  */
+   All trailing whitespace is stripped from the header, and it is
+   zero-terminated.  */
 int
 header_get (struct rbuf *rbuf, char **hdr, enum header_get_flags flags)
 {
@@ -101,11 +101,13 @@
  if (next == '\t' || next == ' ')
continue;
}
- /* The header ends.  */
+
+ /* Strip trailing whitespace.  (*hdr)[i] is the newline;
+decrement I until it points to the last available
+whitespace.  */
+ while (i > 0 && ISSPACE ((*hdr)[i - 1]))
+   --i;
  (*hdr)[i] = '\0';
- /* Get rid of '\r'.  */
- if (i > 0 && (*hdr)[i - 1] == '\r')
-   (*hdr)[i - 1] = '\0';
  break;
}
}

Re: Weird 302 problem with wget 1.7

[John Levon]

On Mon, Jan 14, 2002 at 02:30:54PM +0100, Hrvoje Niksic wrote:

> > moz wget-1.7 188 wget http://www.movementarian.org/oprofile-0.0.8.tar.gz
> > --20:35:51--  http://www.movementarian.org/oprofile-0.0.8.tar.gz
> >=> `oprofile-0.0.8.tar.gz'
> > Connecting to www.movementarian.org:80... connected!
> > HTTP request sent, awaiting response... 302 Moved
> > Location: http://www.movement.uklinux.net/oprofile-0.0.8.tar.gz  [following]
> > --20:35:52--  http://www.movement.uklinux.net/oprofile-0.0.8.tar.gz%20
> >=> `oprofile-0.0.8.tar.gz '
> 
> If you examine this log carefully, you'll notice that their `Location'
> header contains a trailing space.  Wget even reencodes the space as
> %20 to make the URL more readable, but it still retrieves the "wrong"
> URL.

indeed.

> Does someone else know if this is legal?  I guess removing trailing
> spaces from `Location' shouldn't be too harmful.

Someone pointed out :

http://www.ietf.org/rfc/rfc2616.txt

4.2

...

The field-content does not include any leading or trailing LWS:
   linear white space occurring before the first non-whitespace
   character of the field-value or after the last non-whitespace
   character of the field-value. Such leading or trailing LWS MAY be
   removed without changing the semantics of the field value. Any LWS
   that occurs between field-content MAY be replaced with a single SP
   before interpreting the field value or forwarding the message
   downstream.

So wget should always remove it IMHO

regards
john


-- 
"Now why did you have to go and mess up the child's head, so you can get another gold 
waterbed ?
 You fake-hair contact-wearing liposuction carnival exhibit, listen to my rhyme ..."

Re: Weird 302 problem with wget 1.7

[Hrvoje Niksic]

John Levon <[EMAIL PROTECTED]> writes:

> moz wget-1.7 188 wget http://www.movementarian.org/oprofile-0.0.8.tar.gz
> --20:35:51--  http://www.movementarian.org/oprofile-0.0.8.tar.gz
>=> `oprofile-0.0.8.tar.gz'
> Connecting to www.movementarian.org:80... connected!
> HTTP request sent, awaiting response... 302 Moved
> Location: http://www.movement.uklinux.net/oprofile-0.0.8.tar.gz  [following]
> --20:35:52--  http://www.movement.uklinux.net/oprofile-0.0.8.tar.gz%20
>=> `oprofile-0.0.8.tar.gz '

If you examine this log carefully, you'll notice that their `Location'
header contains a trailing space.  Wget even reencodes the space as
%20 to make the URL more readable, but it still retrieves the "wrong"
URL.

Does someone else know if this is legal?  I guess removing trailing
spaces from `Location' shouldn't be too harmful.

Weird 302 problem with wget 1.7

[John Levon]

moz wget-1.7 188 wget http://www.movementarian.org/oprofile-0.0.8.tar.gz
--20:35:51--  http://www.movementarian.org/oprofile-0.0.8.tar.gz
   => `oprofile-0.0.8.tar.gz'
Connecting to www.movementarian.org:80... connected!
HTTP request sent, awaiting response... 302 Moved
Location: http://www.movement.uklinux.net/oprofile-0.0.8.tar.gz  [following]
--20:35:52--  http://www.movement.uklinux.net/oprofile-0.0.8.tar.gz%20
   => `oprofile-0.0.8.tar.gz '
Connecting to www.movement.uklinux.net:80... connected!
HTTP request sent, awaiting response... 404 Not Found
20:35:52 ERROR 404: Not Found.

Where is the space (%20) coming from ? Is it perhaps a bug with my domain
registrar (www.123-reg.co.uk) ?

This is wget 1.7 on RedHat Linux 7.0

I'm not subscribed to the list, please Cc:

thanks
john

-- 
"I went to set up a Yahoo ID for my dog. (Don't ask, but the DOG'S email was 
cluttering my inbox)." 
- Ruthless Advisorette

Re: [Fwd: type=a problem with wget 1.6]

[Maureen O'Drisceoil]

Thanks for your help.  It worked.  This tool is cool!


At 05:59 PM 6/18/01 +0200, you wrote:
>Quoting Maureen O'Drisceoil ([EMAIL PROTECTED]):
>
> >  I'm not sure what part of the debug log is relevant, so here's 
> the
> > whole thing.  Thank you.
> >
> > wget -d ftp://hostname.harvard.edu/CURRENT.URLS2.TXT;type=a
>
>Try the following
>
>wget -d 'ftp://hostname.harvard.edu/CURRENT.URLS2.TXT;type=a'
>
>(one has to escape characters that have a special meaning to your
>shell, as `?', `&', `*', or `;').
>
>-- jan
>
>---+
>   Jan Prikryl  icq | vr|vis center for virtual reality and
>   <[EMAIL PROTECTED]>  83242638 | visualisation http://www.vrvis.at
>---+

Re: [Fwd: type=a problem with wget 1.6]

[Jan Prikryl]

Quoting Maureen O'Drisceoil ([EMAIL PROTECTED]):

>  I'm not sure what part of the debug log is relevant, so here's the 
> whole thing.  Thank you.
> 
> wget -d ftp://hostname.harvard.edu/CURRENT.URLS2.TXT;type=a

Try the following

wget -d 'ftp://hostname.harvard.edu/CURRENT.URLS2.TXT;type=a'

(one has to escape characters that have a special meaning to your
shell, as `?', `&', `*', or `;').

-- jan

---+
  Jan Prikryl  icq | vr|vis center for virtual reality and
  <[EMAIL PROTECTED]>  83242638 | visualisation http://www.vrvis.at
---+

Re: [Fwd: type=a problem with wget 1.6]

[Maureen O'Drisceoil]

 I'm not sure what part of the debug log is relevant, so here's the 
whole thing.  Thank you.

wget -d ftp://hostname.harvard.edu/CURRENT.URLS2.TXT;type=a
DEBUG output created by Wget 1.6 on linux-gnu.

parseurl ("ftp://hostname.harvard.edu/CURRENT.URLS2.TXT";) -> host 
hostname.harvard.edu -> opath CURRENT.URLS2.TXT -> dir  -> file 
CURRENT.URLS2.TXT -> ndir
newpath: /CURRENT.URLS2.TXT
--09:32:01--  ftp://hostname.harvard.edu/CURRENT.URLS2.TXT
=> `CURRENT.URLS2.TXT'
wget: Cannot read /users/maureen/.netrc (Permission denied).
Connecting to hostname.harvard.edu:21... Created fd 3.
connected!
Logging in as anonymous ... 220-FTPD1 IBM FTP CS/390 V2R5 at HOSTNAME, 
09:31:20 on 2001-06-18.
220 Connection will close if idle for more than 120 minutes.

--> USER anonymous

230 'ANONYMOUS' logged on.  Working directory is "ANONYMO.".
Logged in!
==> TYPE I ...
--> TYPE I

200 Representation type is Image
done.  ==> CWD not needed.
==> PORT ... Master socket fd 4 bound.

--> PORT 128,103,151,249,136,1

200 Port request OK.
done.==> RETR CURRENT.URLS2.TXT ...
--> RETR CURRENT.URLS2.TXT

125 Sending data set ANONYMO.CURRENT.URLS2.TXT
done.
Created socket fd 5.

 0K -> .

Closing fd 5
Closing fd 4
250 Transfer completed successfully.
Closing fd 3
09:32:06 (143.19 KB/s) - `CURRENT.URLS2.TXT' saved [5425]

type=a: Command not found.





>Kathryn C/Maureen O <[EMAIL PROTECTED]> writes:
>
> > I recently downloaded and compiled wget 1.6.  I've successfully
> > retrieved documents using the http protocol, but I cannot ftp a file in
> > ascii mode.
> > I read the documentation and found that, "wget also supports the
> > 'type' feature for FTP URLs" and provides the example:
> > ftp://host/directory/file;type=a
> > But this returns the file in binary mode.  If I leave off the
> > type=a, the file is retrieved in binary format also.
> > Has anyone had this problem?
>
>Can you post a debug log?  Also, how exactly do you determine that the
>file was retrieved in binary format?

Re: type=a problem with wget 1.6

[Hrvoje Niksic]

Kathryn C/Maureen O <[EMAIL PROTECTED]> writes:

> I recently downloaded and compiled wget 1.6.  I've successfully
> retrieved documents using the http protocol, but I cannot ftp a file in
> ascii mode.
> I read the documentation and found that, "wget also supports the
> 'type' feature for FTP URLs" and provides the example:
> ftp://host/directory/file;type=a
> But this returns the file in binary mode.  If I leave off the
> type=a, the file is retrieved in binary format also.
> Has anyone had this problem?

Can you post a debug log?  Also, how exactly do you determine that the
file was retrieved in binary format?

type=a problem with wget 1.6

[Kathryn C/Maureen O]

Hi,
I recently downloaded and compiled wget 1.6.  I've successfully
retrieved documents using the http protocol, but I cannot ftp a file in
ascii mode.
I read the documentation and found that, "wget also supports the
'type' feature for FTP URLs" and provides the example:
ftp://host/directory/file;type=a
But this returns the file in binary mode.  If I leave off the
type=a, the file is retrieved in binary format also.
Has anyone had this problem?
Thank you.

Re: problem with wget 1.7

[Hrvoje Niksic]

Arkadiusz Miskiewicz <[EMAIL PROTECTED]> writes:

> please try:
> wget --mirror http://www.ire.pw.edu.pl/zejim/rois/

Thanks for the report.  I believe this patch should fix the problem.

2001-06-14  Hrvoje Niksic  <[EMAIL PROTECTED]>

* recur.c (recursive_retrieve): Also check undesirable_urls with
canonicalized URL.

Index: src/recur.c
===
RCS file: /pack/anoncvs/wget/src/recur.c,v
retrieving revision 1.21
diff -u -r1.21 recur.c
--- src/recur.c 2001/05/27 19:35:09 1.21
+++ src/recur.c 2001/06/14 21:43:21
@@ -381,7 +381,13 @@
}
  xfree (constr);
  constr = xstrdup (u->url);
- string_set_add (undesirable_urls, constr);
+ /* After we have canonicalized the URL, check if we have it
+on the black list. */
+ if (string_set_contains (undesirable_urls, constr))
+   inl = 1;
+ /* This line is bogus. */
+ /*string_set_add (undesirable_urls, constr);*/
+
  if (!inl && !((u->proto == URLFTP) && !this_url_ftp))
if (!opt.spanhost && this_url && !same_host (this_url, constr))
  {

problem with wget 1.7

[Arkadiusz Miskiewicz]

please try:
wget --mirror http://www.ire.pw.edu.pl/zejim/rois/

and wget loops ;( 1.6 version works fine.

-- 
Arkadiusz Miśkiewicz, AM2-6BONE, 1024/3DB19BBD
 http://www.t17.ds.pwr.wroc.pl/~misiek/ipv6/
IPv6 ready PLD/Linux at http://www.pld.org.pl/

Re: problem with wget 1.6 on win98

[Hack Kampbjørn]

Luca wrote:
> 
> > This is "feature" of Windows file system. It doesn't allow '?' in
> > filenames (nor \ / * " < >)
> 
> Thanks, I thought it could have to do with unsafe characters,
> but I couldn't think of any workaround.
> 
> > Use the --output-document=FILE to override the filename. This of
> > course only works for one file and not for recursive download.
> 
> It works, indeed. However, unfortunately this isn't compatible with
> the other options I'll be using (-E -k -K -nh -nd -P AMG -p).
> 
> > If your compiling from the sources I have a quick hack to
> > overcome this.
> 
> No, I'm using Heiko's binaries. But, if this is the only way out,
> I'll go through the compile process myself, after applying your
> hack. Can you tell me more?
> 
 Yes, I can. I submitted it to the patches list just after answering
you. This only handles '?' and the convert-link option won't work for
these so renamed files.

Index: src/url.c
===
RCS file: /pack/anoncvs/wget/src/url.c,v
retrieving revision 1.21.2.1
diff -u -r1.21.2.1 url.c
--- src/url.c   2000/12/17 19:28:20 1.21.2.1
+++ src/url.c   2001/02/03 15:53:24
@@ -1272,16 +1272,17 @@
  file = nfile;
}
 }
-  /* DOS-ish file systems don't like `%' signs in them; we change it
- to `@'.  */
-#ifdef WINDOWS
+  /* Windows file systems don't like `?' signs in them; we change it
+ to `@'. 
+  Note: nor are \ / : * " < > | allowed */
+#if defined(WINDOWS) || defined (__CYGWIN__)
   {
 char *p = file;
 for (p = file; *p; p++)
-  if (*p == '%')
+  if (*p == '?')
*p = '@';
   }
-#endif /* WINDOWS */
+#endif /* WINDOWS or CYGWIN */
 
   /* Check the cases in which the unique extensions are not used:
  1) Clobbering is turned off (-nc).



> Thanks again.
> 
You're welcome

> Best regards,
> Luca

-- 
Med venlig hilsen / Kind regards

Hack Kampbjørn   [EMAIL PROTECTED]
HackLine +45 2031 7799

Re: problem with wget 1.6 on win98

[Luca]

> This is "feature" of Windows file system. It doesn't allow '?' in
> filenames (nor \ / * " < >)

Thanks, I thought it could have to do with unsafe characters,
but I couldn't think of any workaround.

> Use the --output-document=FILE to override the filename. This of
> course only works for one file and not for recursive download.

It works, indeed. However, unfortunately this isn't compatible with
the other options I'll be using (-E -k -K -nh -nd -P AMG -p).

> If your compiling from the sources I have a quick hack to
> overcome this.

No, I'm using Heiko's binaries. But, if this is the only way out,
I'll go through the compile process myself, after applying your
hack. Can you tell me more?

Thanks again.

Best regards,
Luca

Re: problem with wget 1.6 on win98

[Hack Kampbjørn]

Luca wrote:
> 
> I can't download pages from allmusic.com. I must be doing something
> wrong, but I can't figure out what. I tried to fake the user-agent
> but it's still not working. Can you help me? This is the command line
> I used (actually it's a bit more complicated, but I couldn't even get
> this one to work):
> 
> wget -d http://allmusic.com/cg/x.dll?p=amg&sql=Bemhbeheheegi
> 
> [...]
> Cannot write to `x.dll?p=amg&sql=Bemhbeheheegi' (No such file or directory).
> 
> - end of wget output -

This is "feature" of Windows file system. It doesn't allow '?' in
filenames (nor \ / * " < >)

Use the --output-document=FILE to override the filename. This of course
only works for one file and not for recursive download.

If your compiling from the sources I have a quick hack to overcome this.

> 
> Thanks for your attention.
> 
> Best regards,
> Luca

-- 
Med venlig hilsen / Kind regards

Hack Kampbjørn   [EMAIL PROTECTED]
HackLine +45 2031 7799

problem with wget 1.6 on win98

[Luca]

I can't download pages from allmusic.com. I must be doing something
wrong, but I can't figure out what. I tried to fake the user-agent
but it's still not working. Can you help me? This is the command line
I used (actually it's a bit more complicated, but I couldn't even get
this one to work):

wget -d http://allmusic.com/cg/x.dll?p=amg&sql=Bemhbeheheegi

And this is the debug output:

DEBUG output created by Wget 1.6 on Windows.

parseurl ("http://allmusic.com/cg/x.dll?p=amg&sql=Bemhbeheheegi") -> host al
lmusic.com -> opath cg/x.dll?p=amg&sql=Bemhbeheheegi -> dir cg -> file x.dll
?p=amg&sql=Bemhbeheheegi -> ndir cg
newpath: /cg/x.dll?p=amg&sql=Bemhbeheheegi
--15:46:16--  http://allmusic.com/cg/x.dll?p=amg&sql=Bemhbeheheegi
   => `x.dll?p=amg&sql=Bemhbeheheegi'
Connecting to allmusic.com:80... Created fd 48.
connected!
---request begin---
GET /cg/x.dll?p=amg&sql=Bemhbeheheegi HTTP/1.0
User-Agent: Wget/1.6
Host: allmusic.com
Accept: */*

---request end---
HTTP request sent, awaiting response... HTTP/1.0 200 OK
Content-Type: text/html
Content-Length: 5598
Content:


Length: 5,598 [text/html]
x.dll?p=amg&sql=Bemhbeheheegi: No such file or directory
Closing fd 48

Cannot write to `x.dll?p=amg&sql=Bemhbeheheegi' (No such file or directory).

- end of wget output -

Thanks for your attention.

Best regards,
Luca