wget problem

[Juhana Sadeharju]

Hello.
The following document could not be downloaded at all:
  http://www.greyc.ensicaen.fr/~dtschump/greycstoration/

If you succeed, please tell me how. I want all the html file
and the images.

Juhana
-- 
  http://music.columbia.edu/mailman/listinfo/linux-graphics-dev
  for developers of open source graphics software

WGET Bug?

[Nijs, J. de]

Title: WGET Bug?






#

C:\Grabtest\wget.exe -r --tries=3 http://www.xs4all.nl/~npo/ -o C:/Grabtest/Results/log

#

--16:23:02--  http://www.xs4all.nl/%7Enpo/

   => `www.xs4all.nl/~npo/index.html'

Resolving www.xs4all.nl... 194.109.6.92

Connecting to www.xs4all.nl[194.109.6.92]:80... failed: No such file or directory.

Retrying.

#


Is WGET always aspecting a INDEX.HTML al url file for grabbing data from the WWW ?


The most URLs we want to grab are not named as index.html but have other names like:









Is this a problem for WGET by the way?


Kind regardst,

Peter de Nijs
DELTA N.V. afdeling Portfolio Analyse
06-45 57 29 17
06-45 57 29 17


===
Dit e-mailbericht is slechts bedoeld voor gebruik door de geadresseerde.
Dit bericht kan vertrouwelijke informatie bevatten en/of informatie die is beschermd door een beroepsgeheim.
Indien u dit bericht ontvangt terwijl dit niet voor u is bestemd, verzoeken wij u vriendelijk ons hierover
per omgaande te berichten. Bij voorbaat dank!

The information transmitted by e-mail may be privileged or confidential and protected by Law.
If you have received it in error, we would appreciate your notifying us immediately. Thank you!
===

Re: WGET Bug?

[Hrvoje Niksic]

"Nijs, J. de" <[EMAIL PROTECTED]> writes:

> #
> C:\Grabtest\wget.exe -r --tries=3 http://www.xs4all.nl/~npo/ -o
> C:/Grabtest/Results/log
> #
> --16:23:02--  http://www.xs4all.nl/%7Enpo/
>   => `www.xs4all.nl/~npo/index.html'
> Resolving www.xs4all.nl... 194.109.6.92
> Connecting to www.xs4all.nl[194.109.6.92]:80... failed: No such file or
> directory.
> Retrying.
> #
>
> Is WGET always aspecting a INDEX.HTML al url file for grabbing data
> from the WWW ?

No, what you see is the result of two different things:

1. Wget uses "index.html" as the file name when one is missing from
   the URL because it ends with an empty path component.

2. Wget 1.9.1 (and previous versions) doesn't correctly display
   Winsock error messages, such as "connection refused".  The error
   message you're seeing doesn't reflect what really happened.

In this case, only issue #2 is a real bug.  It has been fixed in the
CVS version, which is unfortunately not yet available as a Windows
binary.

NTLM test server

[Hrvoje Niksic]

Is there a test server where one can try out NTLM authentication?  I'm
working on adapting Daniel's code to Wget, and having a test server
would be of great help.

Re: NTLM test server

[Daniel Stenberg]

On Mon, 4 Apr 2005, Hrvoje Niksic wrote:
Is there a test server where one can try out NTLM authentication?  I'm 
working on adapting Daniel's code to Wget, and having a test server would be 
of great help.
Just for your info:
I had friends providing the test servers for both host and proxy 
authentication when I've worked on NTLM code.

Once the basics worked, I could add test cases to the curl test suite. 
Nowadays I can test NTLM on my own with the curl test server. I'm afraid it is 
too specific for curl to be useful for wget.

Also, since the day I provided that code, I've leared a few additional things:
o I had a buffer security problem in the NTLM code, but related to base64
  decode function and that is bound to be different when you adapt the code
  to wget conditions anwyay.
o There was also another less alarming buffer problem with a memset() of 8
  bytes instead of 5. You may have of course already have found and fixed
  this.
o POSTing with NTLM auth is a pain, since NTLM is for connections and thus you
  cannot close the connection without breaking the auth so thus you are (more
  likely than with other multi-pass auth methods) forced to send the full
  request body multiple times.
And probably a little more that I've forgotten to mention now. ;-)
--
 -=- Daniel Stenberg -=- http://daniel.haxx.se -=-
  ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol

Release schedule?

[Jim Wright]

[I previously sent this to [EMAIL PROTECTED] with no response.
This is probably a better forum for this.]


There has obviously been a great deal of work recently on wget, and
a new release seems to be on the horizon.  I have some patches to add
functionality, but I've not taken the time to clean them up and send
it in.  But of course, I really want to see it in the next release.
So... Is there a plan for the upcoming release?  When might a feature
freeze take place?

Thanks,
Jim

Re: NTLM test server

[Hrvoje Niksic]

Daniel Stenberg <[EMAIL PROTECTED]> writes:

> I had friends providing the test servers for both host and proxy
> authentication when I've worked on NTLM code.

It's a shame that those test servers are no longer available.  I don't
think it will be possible to finish the NTLM code without some sort of
test bed.

> o I had a buffer security problem in the NTLM code, but related to
>base64 decode function and that is bound to be different when you
>adapt the code to wget conditions anwyay.

I've seen notice of it and fixed allocation of BUFFER accordingly.

> o There was also another less alarming buffer problem with a
>memset() of 8 bytes instead of 5. You may have of course already
>have found and fixed this.

I've now changed this, thanks.

> o POSTing with NTLM auth is a pain, since NTLM is for connections and thus you
>cannot close the connection without breaking the auth so thus you
>are (more likely than with other multi-pass auth methods) forced
>to send the full request body multiple times.

I guess there's no avoiding that.

wget 1.10 alpha 1

[Mauro Tortonesi]

i have just released the first alpha 1 of wget 1.10:

ftp://ftp.deepspace6.net/pub/ds6/sources/wget/wget-1.10-alpha1.tar.gz
ftp://ftp.deepspace6.net/pub/ds6/sources/wget/wget-1.10-alpha1.tar.bz2

with LFS support and several other interesting features and bugfixes. as 
always, testers are very welcome ;-)

BTW, since i want to release wget 1.10 ASAP, i am scheduling the official 
feature freeze for wget 1.10 for be sunday, april 9th. if nothing goes wrong, 
i am planning to release wget 1.10 before the end of april (and possibly in a 
couple of weeks).

-- 
Aequam memento rebus in arduis servare mentem...

Mauro Tortonesi

University of Ferrara - Dept. of Eng.http://www.ing.unife.it
Institute of Human & Machine Cognition   http://www.ihmc.us
Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net
Ferrara Linux User Group http://www.ferrara.linux.it

Re: Release schedule?

[Mauro Tortonesi]

On Monday 04 April 2005 04:45 pm, you wrote:
> [I previously sent this to [EMAIL PROTECTED] with no response.
> This is probably a better forum for this.]
>
>
> There has obviously been a great deal of work recently on wget, and
> a new release seems to be on the horizon.  I have some patches to add
> functionality, but I've not taken the time to clean them up and send
> it in.  But of course, I really want to see it in the next release.
> So... Is there a plan for the upcoming release?  When might a feature
> freeze take place?

sorry. it's all my fault. i have been ***EXTREMELY*** busy lately working on a 
research project and i haven't been working on wget as much as i wanted. i 
cannot but apologize. i'll try to do my best to catch up ASAP.

anyway, i just released the first alpha 1 of wget 1.10:

ftp://ftp.deepspace6.net/pub/ds6/sources/wget/wget-1.10-alpha1.tar.gz
ftp://ftp.deepspace6.net/pub/ds6/sources/wget/wget-1.10-alpha1.tar.bz2

and i was thinking about releasing wget before the end of april, after we have 
performed some tests on the 1.10 code.

so, the official feature freeze would be sunday, april 9th. but, since the 
recenly integrated LFS feature seems to be very useful and there's not been a 
release of wget in ages, unless we have a major reason (like fixing a serious 
bug or integrating an extremely cool and widely used feature) i think we 
should really focus on testing and bugfixing at this point.

WRT your patches, please post them on the [EMAIL PROTECTED] mailing list 
with some comments on what they do and especially why you think they are 
needed. don't bother to clean the code yet, as they might be rejected or 
cleaned up by other developers.

-- 
Aequam memento rebus in arduis servare mentem...

Mauro Tortonesi

University of Ferrara - Dept. of Eng.http://www.ing.unife.it
Institute of Human & Machine Cognition   http://www.ihmc.us
Deep Space 6 - IPv6 for Linuxhttp://www.deepspace6.net
Ferrara Linux User Group http://www.ferrara.linux.it