Assertion failed in wget-1.8.1, progress-bar related.

[DervishD]

Hello wget hackers :)

While mirroring Debian I killed wget, and when resuming the
mirroring one of the files in the remote site 'shrunk' (correctly, it
is updated daily). The problem is that, being the new size smaller
than the old, wget doesn two things wrong:

- First, it doesn't retrieve the file *entirely*, as it should do
(IMHO) if the remote file is smaller than the local copy.
- Second, it fails with: "progress.c:552: create_image: Assertion `percentage <= 
100' failed."
fail that is perfectly logical: the computed percentage, being the
remote file smaller, is bigger than 100%.

Please note that this assertion is correct, and it's violated
just as a consequence of the above noted behaviour. If wget gets
entirely files smaller or newer than the local copy, the assertion
will be respected.

If you need more info, please don't doubt to ask and I'll try to
provide all information I can.

Thanks a lot :)
Raúl Núñez de Arenas Coronado

Bug, feature or my fault?

[DervishD]

Hi all :))

This is my first message on this list and as usual is a call for
help ;) Well, the question is that I don't know if this is a bug
(haven't look at the sources yet) and I can't find nothing in the
documentation.

So, prior to send a bug report, I want to make sure this is a bug
and not my fault O:)

Let's go to the matter: when I download, thru FTP, some
hierarchy, the spaces are translated as '%20'. Nothing wrong with
that, of course, it is a safe behaviour. In fact, when files with
spaces in their names are saved, the %20 is translated back to a real
space. But its not the case for directóries, that are created with
%20 treated as three characters, literally the sequence "%20". An
example, server side:

A directory/With a file in it.txt

while downloading we can see:

A%20directory

and

With%20a%20file%20in%20it.txt

and locally they are stored as:

A%20directory/With a file in it.txt

This looks like a bug for me... For the details, I'm using wget
version 1.8.2, and the only options passed are '-c', '-r' and '-nH'
(and, of course, the URL ;))).

Any help? Is this a bug?

Thanks a lot in advance :)

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: J K wants to unsubscribe ?

[DervishD]

Hi Peter :)

 * Peter Skye <[EMAIL PROTECTED]> dixit:
> My two recent posts to this list were responded to, apparently
> automatically, by "J K" whose email address is "[EMAIL PROTECTED]".

Yes, is a poor brain-damaged retarded, whose posts on other lists
are of this style: "gimme information about ". He has
forgotten his 'p & q' obviously. His lack of politeness, together
with his lack of intelligence lead to posts like the one you quote
below:
 
> The message both times from J K was "F*** YOU"
> (the asterisk replacements are mine).

I've tried to f*** myself, but I was unable so I didn't answer
him, but really I received the message.
 
> Does J K want to unsubscribe from the list but can't figure out how to
> do so?  Have others received this message?

Hi cannot unsubscribe from the list, because you need at least
some rudiments about reading and writing, and he hasn't. So the
bitter anger and the effin message... In fact, this was the only
response to the nearly-bug-report I post a couple of days ago, and I
just thought that this list had a bad day XDD

If that poor fellow keeps on suggesting that kind of pleasure
behaviour to the members of the list, someone should unsubscribe for
him, since he is unable... Poor boy...

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Bug, feature or my fault?

[DervishD]

Hi all :))

After asking in the wget list (with no success), and after having
a look at the sources (a *little* look), I think that this is a bug,
so I've decided to report here.
 
Let's go to the matter: when I download, thru FTP, some
hierarchy, the spaces are translated as '%20'. Nothing wrong with
that, of course, it is a safe behaviour. In fact, when files with
spaces in their names are saved, the %20 is translated back to a real
space. But its not the case for directóries, that are created with
%20 treated as three characters, literally the sequence "%20". An
example, server side:

A directory/With a file in it.txt

while downloading we can see:

A%20directory

and

With%20a%20file%20in%20it.txt

and locally they are stored as:

A%20directory/With a file in it.txt

Moreover, if the entire file name is passed to wget in the URL,
the entire filename has the annoying '%20' instead of unquoted
spaces. It seems that only the part that wget 'discovers' when
crawling is unquoted.

This looks like a bug for me... For the details, I'm using wget
version 1.8.2, and the only options passed are '-c', '-r' and '-nH'
(and, of course, the URL ;))), and obviously the wgetrc settings... I
don't think that any option is affecting this. The complete command
line is:

$ wget -c -r -nH "ftp://user:[EMAIL PROTECTED]/A Directory/*"
 
Any help? Is this a bug? If you want to test I can provide you
with a test account in the server that is causing the problem (I
don't know of any other site that has directories and filenames with
spaces or other quoteable chars), any output from 'wget -d', my
global wgetrc or any other thing you need.

The bug seems to be in the handling of the struct url that is
passed to the ftp module. It seems that the unquoted members (dir and
file) are not used to form the filename that will be stored on disk
:??? Currently I have not much spare time to look for the bug, but if
I can test something you can count on it.

Thanks a lot in advance :)

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: upcoming new wget version

[DervishD]

Hi Jochen :)

 * Jochen Roderburg <[EMAIL PROTECTED]> dixit:
> Question: Is the often discussed *feature* in version 1.8.x meanwhile 
> repaired, that special characters in local filenames are url-encoded?
> That was the main reason for me that I never started using that version ;-)

And that is the only wget bug that keeps hitting me again and
again :((( specially when I download files with spaces in it. I
posted a complete and detailed bug report a couple of months ago.

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Portable representation of large integers

[DervishD]

Hi Hrvoje :))

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> There have been several attempts to fix this:
[...]
> * In its own patches, Debian introduced the use of large file APIs and
>   `long long'.  While that's perfectly fine for Debian, it is not
>   portable.  Neither the large file API nor `long long' are
>   universally available, and both need thorough configure checking.

Yes, you're true, but... How about using C99 large integer types
(intmax_t and family)? I know, is not universally available, but if
you support obsolete systems IMHO you should do in a last resort way.
I mean, use large file API or intmax_t on every system that supports
it (all systems with a not-so-recent GNU C compiler), long long on
the rest and long as the last resort. It's not very complicated to do
so in autoshi^H^H^Hconf, and wget will support really huge files in
most of systems (those with a GNU C compiler and those with long long
support), and normal files on the rest.

> Of those two issues, choosing and using the numeric type is the hard
> one.  Autoconf helps only to an extent -- even if you define your own
> `large_number_t' typedef, which is either `long' or `long long', the
> question remains how to print that number.

If you use intmax_t, the problem is solved because C99 has a
%-format for them. If you use long long and gcc, the same happens
AFAIK. The problem is that you don't have an universal format for all
those types... You should write a printf wrapper or a special
function for printing those numbers.

> 1. Concatenation of adjacent string literals is an ANSI feature and
>would break pre-ANSI compilers.

You're true again, but you can wrap it in a macro and use '##'.
Or you can stop supporting pre-ANSI compilers. How many users will
this affect (more or less).

> 2. It breaks gettext.  With translation support, the above code would
>look like this:

Yes, gettext is far from perfect : The solution is to do the
conditional preprocessing above directly on the code, so you have
three (or more, as needed) printf lines with legal-gettext code in
them:

#ifdef CRAP_SYSTEM
printf(_("Whatever %d\n"), num);
#elif defined A_BIT_BETTER_SYSTEM
printf(_("Whatever %lld\n", num);
#else
/* We have a good system */
printf(_("Whatever %j\n", num);
#endif

Yes, looks ugly, is crap, but... 

> The bottom line is, I really don't know how to solve this portably.

IMHO, the best solution is to use a big unsigned type for sums,
totals, sizes to print, etc... and use a large file API for reading
and write large files. If the large file API doesn't exist, use off_t
anyway. Provide a function for printing off_t's and just make sure
that the type for sums is larger than off_t if possible, otherwise
use off_t too (I suppose that autoconf can help you in this).

Normally I use 'size_t' or 'uintmax_t' for sizes to print, sums,
totals, etc... and off_t for files, because an off_t, being signed,
will be shorter than the uintmax type for sure, although you can
always use bignum libraries. The bignum libraries (well, pick up just
the code for additions, for example) will help you with half of the
problem. The other half, reading/writing large files, must be
provided by the system. Just define _LARGEFILE64 (or whatever is the
macro called in SuSv3 or POSIX) and use off_t and an small function
for printing them (although off_t is opaque and if you want full
portability you should not use it directly...).

That's the better I can get, because when I wrote portable code,
by portable I understand 'according to standards'. For me that means,
in that order: SuSv3, POSIX, C99, C89, stop. No pre-ANSI and no brain
damaged compilers.

> Does anyone know how widely ported software deals with large files?

No idea, sorry :((( And thanks for the work you're doing with
wget, we all benefit from.

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Portable representation of large integers

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> Using #ifdefs to switch between %d/%lld/%j *is* completely portable,
> but it requires three translations for each message.  The translators
> would feast on my flesh, howling at the moonlight.

Sure ;)) Anyway, it will happen in just a couple of places. I
mean, it's not like triplicating the number of messages...

> Hmm.  How about preprocessing the formats before passing them to
> printf?

That will help with the translation, obviously, but as I said
above, if the number of affected messages is short...

> > That's the better I can get, because when I wrote portable code, by
> > portable I understand 'according to standards'. For me that means,
> > in that order: SuSv3, POSIX, C99, C89, stop. No pre-ANSI and no
> > brain damaged compilers.
> I understand your position -- it's perfectly valid, especially when
> you have the privilege of working on a system that supports all
> those standards well.  But many people don't, and Wget (along with
> most GNU software of the era) was written to work for them as well. 

Of course. Moreover, my software is not used for as many users as
wget is. In fact, wget is the standard for URL retrieving under UNIX
(together with curl). It really worths the effort making fully
portable and runnable in as many systems as possible.

I like portability, and in fact I try my best to write my
software portably, and that include, for example, tweaking a bit with
types, not assuming anything, etc... but that doesn't include
bad-behaviour systems: weird libc's, systems where off_t should be 64
but due to some lazy programmer it's 32 by mistake, etc... In other
words, I adhere to standards (even old ones if necessary), but I
don't go out from them for supporting weird systems (pre-ANSI, for
example). I have that vision of portability mainly because non
standard systems have been biting me until I started to use Linux
(1995 or so...).

> For me, portability is not about adhering to standards,
> it's about making programs work in a wide range of environments,
> some of which differ from yours.

I think that standards are good, so my opinion. But I'm with you:
this is not applicable for wget. You are so kind for supporting such
a variety of systems. I wouldn't support them, but fortunately for
wget users, I'm not the maintainer :)) I must confess that lately my
vision of portability have been reduced to 'can compile under Linux
or at least under a SuSv3 or POSIX system'. Please note that this is
not arbitrary: just like you want wget to be able to run in a wide
range of environment, I don't like to punish performance or power of
my software just to make it run in a system I don't feel like
supporting. Anyway, I'm not radical with this: each piece of software
must be dealt with in its unique way.

Just in case you need, I have the entire ANSI C 99 standard, even
with the rationale, so if you don't have it and want to know
something, please feel free to ask. I'll try to help as much as I
can.
 
> Thanks for your suggestions.

Thanks to you for considering and for wget, really. It saves me a
lot of effort and time when dowloading Linux distros :)))

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Wget 1.9-beta1 is available for testing

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> http://fly.srk.fer.hr/~hniksic/wget/wget-1.9-beta1.tar.gz

I've got and tested it, and with NO wgetrc (it happens the same
with my own wgetrc, but I tested clean just in case), the problem
with the quoting still exists:

$wget -r -c -nH "ftp://user:[EMAIL PROTECTED]/Music/Joe Hisaishi"

--15:22:53--  ftp://user:[EMAIL PROTECTED]/Music/Joe%20Hisaishi
   => `Music/.listing'
Resolving david... 213.97.199.90
Connecting to david[213.97.199.90]:21... connected.
Logging in as user ... Logged in!
==> SYST ... done.==> PWD ... done.
==> TYPE I ... done.  ==> CWD /Music ... done.
==> PORT ... done.==> LIST ... done.

0K   7.92 KB/s

15:22:55 (7.92 KB/s) - `Music/.listing' saved [347]

Removed `Music/.listing'.
--15:22:55--  ftp://user:[EMAIL PROTECTED]/Music%2fJoe%20Hisaishi/Joe%20Hisaishi
   => `Music%2FJoe Hisaishi/.listing'
==> CWD /Music/Joe Hisaishi ... done.
==> PORT ... done.==> LIST ... done.

0K .10.28 KB/s

15:22:56 (10.28 KB/s) - `Music%2FJoe Hisaishi/.listing' saved [1286]

Removed `Music%2FJoe Hisaishi/.listing'.
--15:22:56--  ftp://user:[EMAIL 
PROTECTED]/Music%2fJoe%20Hisaishi%2fBrother/Joe%20Hisaishi
   => `Music%2FJoe Hisaishi%2FBrother/.listing'
==> CWD /Music/Joe Hisaishi/Brother ... done.
==> PORT ... done.==> LIST ... done.

0K . 9.59 KB/s

15:22:57 (9.59 KB/s) - `Music%2FJoe Hisaishi%2FBrother/.listing' saved [1200]

Removed `Music%2FJoe Hisaishi%2FBrother/.listing'.
--15:22:57--  ftp://user:[EMAIL 
PROTECTED]/Music%2fJoe%20Hisaishi%2fBrother/01%20-%20Drifter%20...%20In%20LAX.mp3
   => `Music%2FJoe Hisaishi%2FBrother/01 - Drifter ... In LAX.mp3'
==> CWD not required.
==> PORT ... done.==> RETR 01 - Drifter ... In LAX.mp3 ... done.
Length: 4,200,538

0K .. .. .. ..


At this point I have the following directory entries:

Music
Music%2FJoeHisaishi
Music%2FJoeHisaishi%2FBrother

Inside the last dir there is the file 01 - Drifter..., correct,
without quoting.

As you can see, the slash is quoted and this causes the problem.
In fact, is worse than before the patch, because before the only
damaged part was the 'Joe Hisaishi' one, namely mangled as
'Joe%20Hisaishi'. In fact, all items provided in the command line are
quoted, but happens that 'Music' is not quotable (well, it hasn't
wrong characters...). Am I doing something wrong?

My platform is a do-it-yourself Linux box, running kernel 2.4.21
and libc 2.2.4, and if you need an username and a password for
testing yourself I can provide it, just ask.

Apart from this, the release seems to be pretty stable. I've
tested it against my own ftp server (the one in the example...),
turning it down in the middle of a transfer and things like those
and it seems to work right.

If you want me to test something more for this release, just
tell and I'll do. Thanks for wget :))

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Wget 1.9-beta1 is available for testing

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> Thanks for the detailed bug report.  Although it doesn't look that
> way, this problem is nothing but a simple oversight.

OK, patch applied and working! Now it downloads correctly all
files, quote appropriately and all works smooth :))

Thanks a lot for the patch.

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: wget bug

[DervishD]

Hi Jack :)

 * Jack Pavlovsky <[EMAIL PROTECTED]> dixit:
> It's probably a bug:
> bug: when downloading 
> wget -mirror ftp://somehost.org/somepath/3acv14~anivcd.mpg, 
>  wget saves it as-is, but when downloading
> wget ftp://somehost.org/somepath/3*, wget saves the files as 
> 3acv14%7Eanivcd.mpg

Yes, it *was* a bug. The lastest prerelease has it fixed. Don't
know if the tarball has the latest patches, ask Hvroje. But if you
are not in a hurry, just wait for 1.9 to be released.

> The human knowledge belongs to the world

True ;))

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Wget 1.9-beta2 is available for testing

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> This beta includes several important bug fixes since 1.9-beta1, most
> notably the fix for correct file name quoting with recursive FTP
> downloads.

That works, at least for me. I've tested with the ftp repository
that previously failed and now it works (well, it worked with the
patch you sent after 1.9-beta1...).

Do you want me to test anything more? The rest of my normal use
with wget works correctly.

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: How do you pronounce Hrvoje?

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> > Can you give us linguistically challenged Americans a phonetic
> > rendition of your name?
> It's not easy to describe because of the phonems and concepts not
> present in the English language.  You'll probably regret having asked.
[...]

Very interesting!. I really didn't know how to pronounce your
name, I even had problems writing it at first ;))) Nice to know how
to pronounce it. I'm spanish, so I don't exactly know if I got all
sounds correctly, but I think I have a pretty good approximation.
Thanks ;))

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Wget 1.9 has been released

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> I've announced the 1.9 release on freshmeat

Thanks a lot for your work, Hrvoje, and thanks for Wget ;))
 
Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Naughty make install.info, naugthy, bad boy...

[DervishD]

Hi Hrvoje :))

I've downloaded and installed wget 1.9 without problems, but when
I install something seamlessly, I insist on messing around until I
break something...

The matter is that if you delete 'wget.info' to force recreation,
and your makeinfo is more or less recent, you *don't* have
wget.info-[0-9] files, since new texinfo's have the default --split-size
limit raised from 50k to 300k. So, when poor little bash run the
following:

for file in wget.info wget.info-*[0-9]
do
install -c -m 644 $file /destination/dir/$file
done

the first iteration 'file' contains 'wget.info', but the second
one it contains 'wget.info-*[0-9]', a valid shell pattern that leads
to the following:

install -c -m 644 /destination/dir/wget.info-1 \
/destination/dir/wget.info-2 \
/destination/dir/wget.info-3 \
/destination/dir/wget.info-4

I've split the lines for readability... The problem is that
'install' barfs saying that you're installing multiple files and the
last one is not a directory.

If you add double quotes around '$$file' in that part, the error
just changes, from 'multiple-files-last-not-dir' to 'cannot stat that
file'. The problem is in the globbing... I've solved this in the past
using '$(wildcard)' for the globbing and not using wildcards in the
'for' loops, but it's a bit complex, I must confess. You can do with
this easier solution:

for file in wget.info*
...

Since you're sure that at least wget.info exists, the globbing
won't expand to an empty string (which will make bash barf), so
you're safe and you don't depend on wget.info-whatever existing or
not, and you don't run the risk of a glob pattern going unexpanded in
the 'install' command line.

I'm not sure if this is a good solution at all, as I've said
above this is not the solution I've done in the past, so I don't
provide a patch. It's not urgent, and I want to hear alternatives
prior to preparing a patch.

And thanks again to Hrvoje and all other people who has
collaborated with wget: good work :))

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Naughty make install.info, naugthy, bad boy...

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> > I've downloaded and installed wget 1.9 without problems, but when I
> > install something seamlessly, I insist on messing around until I
> > break something...
> :-)

The problem is that I do that with my *own* software, too XDD
 
> > The matter is that if you delete 'wget.info' to force recreation,
> > and your makeinfo is more or less recent, you *don't* have
> > wget.info-[0-9] files, since new texinfo's have the default
> > --split-size limit raised from 50k to 300k.
> That must be a Makeinfo 4.5 thing.  I'm still using 4.3, which has the
> split limit unchanged.

In fact I think that it is a 4.6 thing. But it should not matter
at all, the only difference is how many info files are generated.
 
> I think I originally used the more complex forms because I wanted to
> avoid matching something like wget.info.bak.  I'm not sure if there
> was a specific reason for this or if I was just being extra-careful.

You're right, the simpler glob (wget.info*) will match any
garbage after the '.info' part :((( Definitely it's not a good idea.
 
> for file in wget.info wget.info-*[0-9]
> do
>   test -f "$file" && install -c -m 644 "$file" ...
> done

This should do, since '$file' won't be never empty : It must
be done in *both* parts of the surrounding 'if-fi' clause...
 
> (Of course, it would use $$file and such in actual Makefile, but you
> get the picture.)

Yes, yes... It's a long story but I've dealt a lot with
makefiles... In fact, the solution I was talking about (using the
'wildcard' function of GNU make, avoid globbing in for loops, etc...)
is caused by an special generated makefile to avoid an empty glob
pattern in the 'for' loop. Here is not needed at all: I was blind and
I even didn't thought about the simpler solution you provide O:))
 
> That way we retain the strictness of only matching wget.info and
> wget.info-, but avoid problems when only wget.info is
> actually generated.

Right :)) If you want I can prepare the patch for you, containing
too a typo in the documentation. BTW, in the documentation there is
no information about that new --retry-connrefused (at least I haven't
found it) and obviously no mention about any rcfile equivalent, am I
missing something or I should wait for 1.9.1?

Thanks a lot for wget, as always (I use it a lot), and if you
want me to prepare the patch, just tell.

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Naughty make install.info, naugthy, bad boy...

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> > Right :)) If you want I can prepare the patch for you, containing
> > too a typo in the documentation.
> I think I'll modify the Makefile.  A patch that fixes (or points out)
> the typo in the documentation would be appreciated, though.

Attached. Is a one-liner, though... I haven't read the docs up to
the point of discovering more typos. I'm a typo expert: in my
documentations I put a lot, for people to enjoy discovering them XDD

Just one more thing: reading the info, I don't understand the
precedence of the timeout settings. For example, I put this in my
global rc:

connect_timeout=60 
dns_timeout=0
read_timeout=600
timeout=0

Notice the last line. Will does that mean that *all* timeouts are
disabled, even though I state explicitly the contrary on the above
lines? Does a '0' in 'dns_timeout' disable DNS timeouts (or let the
timeout be the one chosen by the system, that is right for me, too)?.

This links with my other question: I want to put *all* wgetrc
commands in my wgetrc file, with their defaults if I haven't change
them, just for reference (is easier for me). The problem is that I
don't know what are the defaults of some entries, or even if I can
put things like 'accept=', without a value, in the rc file. Could you
help me on this?

Thanks a lot in advance :)

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/
diff -Naur wget-1.9.orig/doc/wget.texi wget-1.9/doc/wget.texi
--- wget-1.9.orig/doc/wget.texi 2003-10-07 01:12:37.0 +0200
+++ wget-1.9/doc/wget.texi  2003-10-26 12:46:12.0 +0100
@@ -2183,7 +2183,7 @@
 @item cache = on/off
 When set to off, disallow server-caching.  See the @samp{-C} option.
 
[EMAIL PROTECTED] convert links = on/off
[EMAIL PROTECTED] convert_links = on/off
 Convert non-relative links locally.  The same as @samp{-k}.
 
 @item cookies = on/off

Re: Naughty make install.info, naugthy, bad boy...

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> > connect_timeout=60 
> > dns_timeout=0
> > read_timeout=600
> > timeout=0
> >
> > Notice the last line. Will does that mean that *all* timeouts are
> > disabled,
> Yes.  The "timeout" option is a shortcut for specifying all the other
> options.

What I mean is: if I don't specify 'timeout' in my wgetrc it has
the default value ('900', AFAIK), but it is ignored? It shouldn't,
IMHO, it's a weird behaviour, that way I cannot specify commands in
my wgetrc with their default values and have the default behaviour
(if I put timeout=900 then *all* timeouts are 900, no matter if I set
dns_timeout to '0' in wgetrc). I'm afraid I'm missing something...
 
> > This links with my other question: I want to put *all* wgetrc
> > commands in my wgetrc file, with their defaults if I haven't change
> > them, just for reference (is easier for me). The problem is that I
> > don't know what are the defaults of some entries, or even if I can
> > put things like 'accept=', without a value, in the rc file. Could
> > you help me on this?
> I think you should be able to use `accept='.  As for the defaults,
> take a look at the defaults() function in src/init.c.

Thanks a lot :))) 

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Adding port range option to `--bind-address'?

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> Does anyone think it would be useful to add the ability to specify
> port ranges with `--bind-address'?

Reading the explanation from Turi it seems reasonable, and I
don't think that such a feature impose any burden on wget :?

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Naughty make install.info, naugthy, bad boy...

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> >> Yes.  The "timeout" option is a shortcut for specifying all the
> >> other options.
> > What I mean is: if I don't specify 'timeout' in my wgetrc it has the
> > default value ('900', AFAIK), but it is ignored?
> I'm afraid I don't understand this question.  Is what ignored?  You
> say you don't specify it -- how can it be ignored then?

Sorry, I'm afraid my english is somewhat poor :((( What I mean is
that the option 'timeout' has a default value if it is not specified
in wgetrc, true?. But if you specify any *_timeout option, then that
default is not used for the options you specify, true again? Then
that value is being ignored, isn't it?. But if you specify
'timeout=900' in the wgetrc, then any other *_timeout option is
ignored. The fact is that I'm using 'timeout' with its default value
but the behaviour of wget is not the same as if I was not using
'timeout'. I hope I've explained correctly this time O:))

> > it's a weird behaviour, that way I cannot specify commands in my
> > wgetrc with their default values and have the default behaviour
> Sure you can, just don't use "timeout".  It's meant as a simple way to
> set (or unset) all timeouts.

OK. I don't like that behaviour, but I understand the use of
'timeout', thanks :))

> > (if I put timeout=900 then *all* timeouts are 900, no matter if I
> > set dns_timeout to '0' in wgetrc).
> I think this is a case of
> "Doctor, doctor, it hurts when I do that."
> "Don't do that, then."

X''

> Seriously, if you don't want to change dns_timeout, then don't use the
> timeout option.

OK, I take note ;)

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Naughty make install.info, naugthy, bad boy...

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> > Sorry, I'm afraid my english is somewhat poor :((( What I mean is
> > that the option 'timeout' has a default value if it is not specified
> > in wgetrc, true?.
> Not entirely.  The "timeout" does not translate to a single variable,
> but to three variables.  Changing "timeout" modifies all three.

OK, I got it ;))

> > OK. I don't like that behaviour, but I understand the use of
> > 'timeout', thanks :))
> How would you have it behave?

This is only my opinion, but I would like something like this: if
you set 'timeout', it only applies to any *_timeout option not
specified in the wgetrc, not to all of them, specified or not, like
it is now. Or even more complicated: apply timeout to *_timeout
options not specified in the wgetrc or to any whose value is '-1'. As
you can see, it is better as it is now O:))

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Problem with 'page_requisites'

[DervishD]

Hi all :)

I have modified my wgetrc, and now I'm not able to retrieve a
simple file thru FTP with wget 1.9. The problem is the command
'page_requisites' in the wgetrc. I want wget to automatically
retrieve page requisites, and I thought that this setting was ignored
for FTP or when retrieving non-html files. Well, if I set it to 'on',
I must activate 'recursive' too in order to be able to retrieve
simple non-html files thru FTP :(

What am I doing wrong? Why this setting makes retrieval
impossible? Thanks a lot :)))

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Problem with 'page_requisites'

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:

> Raúl, I'm not sure I understand what exactly you are trying to do.  I
> understand that you set page_requisites to on, but I don't see why you
> feel you must activate recursion to download non-HTML files through
> ftp.

Because otherwise it doesn't work ;))
 
> Could you be more specific in how you invoke Wget, what you expect it
> to do, and what it's doing instead?

I was trying to download Fedora (the new RedHat):

wget -nH -nd 
"ftp://ftp.redhat.com/pub/redhat/linux/beta/severn/en/iso/i386/severn-i386-disc2.iso";

(no, I'm not stupid - well, only a bit - I'm downloading it for a
friend and another friend has downloaded part 1...). If I do the
above with 'page_requisites=on', it doesn't work. If I set it to
'off' or change the command line to 'wget -r -nH -nd...', it works
again and starts downloading. If you want my wgetrc just tell, but if
I delete entirely the wgetrc and do the above with '-p', it doesn't
work except if I pass '-r' too.

Thanks, Hrvoje, for your help. I think it's something I'm doing
bad... I haven't build wgetrc with debugging support, but I can do if
it helps.

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Problem with 'page_requisites'

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> So what you're saying is that `wget -p ftp://server/dir/file' doesn't
> work.  I guess I've never tested that.  I'll look into it, thanks for
> the report.

But take into account that wget -p -r ftp://server/dir/file *works*

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Problem with 'page_requisites'

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> I think I've now fixed this problem.  However, `-p' still implies
> creating directory hierarchies, which is mildly annoying.

I find it reasonable (more or less...).
 
> IMO `-p' should be reworked to function like Mozilla's "save web page,
> complete".  I.e. it should save "foo.html" along with "foo_html_files"
> where all the link contents should reside.  If no sublinks exist, the
> directory doesn't get created, which would be the typical case for
> FTP.

OK. Then, just curiosity, why it doesn't work for FTP files and
why it works for FTP files when passing '-r'?

Thanks for fixing ;)

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Problem with 'page_requisites'

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> > OK. Then, just curiosity, why it doesn't work for FTP files and why
> > it works for FTP files when passing '-r'?
> Because `-p' is implemented as a special "leaf" recursion mode that
> sets maximum recursion depth to 0.

OK, ok. I tried with '-r' because in the docs I read that '-p'
was related with recursion, so... Now this confirms my first guess.

Thanks for clearing that :)

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: The libtool situation

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> Is it me, or is there something really strange going on with libtool?
> I tried updating Wget's libtool, but libtool 1.5 is *still*
> unavailable from the GNU FTP site and its mirrors.  It points to a
> "BACK-RSN" type of file that explains how they've been compromised,
> and so on.

The problem affects more packages than just libtool, but AFAIK
the 'compromise' happened a long time ago, and the software has not
been yet replaced :?
 
Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Simplification of `po/' outlined

[DervishD]

Hi Hrvoje :)

I don't use i18n anyway, but...

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> 1. If the gettext command-line tools (specifically msgfmt) are
>unavailable at configure time, don't make the `po' subdirectory.

Nice.
 
> 2. Build with NLS support if the gettext() is available (and
>--disable-nls is not specified), regardless of the above.  That way
>a sysadmin can install Wget first and the command-line and the PO
>files later, and everything will still work.

Perfect. That way any sysadmin can change his/her mind after
installing and rebuilding is not needed.

> 4. Create a normal and fairly simple po/Makefile.in instead of the
>current Makefile.in.in monstrosity.

Very good idea...
 
> What do you think?

The solution is perfect for me. I don't use i18n, as I told, but
I think is a good idea to be able to change my mind in the future.
This solution solves that issue and simplifies Wget tarball.

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Retire little-used one-letter options

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> This patch retires the options -C, -g, -G, and -s.  I hope that this
> won't step on too many toes, as all these options are fairly obscure.

I can see no problem, since all those options are rarely used,
and a bit more of typing didn't kill anyone in the past, IMHO...

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Relevance of MACHINES

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> I'm not sure how to manage the MACHINES file in the distribution.  As
> very few people keep testing the old operating systems documented in
> MACHINES, it's impossible to guarantee that new versions of Wget will
> compile or work on them.

As a developer and as an user, I prefer no documentation rather
than an obsolete, and maybe wrong, one.
 
> One way to fix this would be to accopmany the OS entries in MACHINES
> with the version of Wget that they apply to.  But the problem is that,
> as each version is released, you will only see which machines the
> *previous* versions worked on.

But you have that anyway: since you are releasing a new version,
you cannot know if it will compile or install in any particular arch,
unless you can see the future ;)

> Maybe the current form of MACHINES is simply not relevant any more?

Not for me, certainly. IMHO, you should get rid of MACHINES. If
someones wants to install wget, is not a great effort to give a try.
In the vast majority of cases, it will build and install smoothly,
and if it has a problem, the mailing list address is more useful than
a file saying that you should have a i686-pc-gnu-linux instead you
risc-acorn or whatever.

I find a very sensible idea to eliminate MACHINES. I don't think
it has any real use now if you cannot track down versions, and
anyway it won't give you information for the current version...

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/

Re: Relevance of MACHINES

[DervishD]

Hi Hrvoje :)

 * Hrvoje Niksic <[EMAIL PROTECTED]> dixit:
> >> One way to fix this would be to accopmany the OS entries in MACHINES
> >> with the version of Wget that they apply to.  But the problem is that,
> >> as each version is released, you will only see which machines the
> >> *previous* versions worked on.
> > But you have that anyway: since you are releasing a new version,
> > you cannot know if it will compile or install in any particular arch,
> > unless you can see the future ;)
> I always test the new release on the machines I have access to,
> currently Linux and Solaris.  The "release candidate" process makes
> sure that the release compiles on the architectures that people on
> this list care about (e.g. Windows, but possibly others).  You don't
> need a time machine for that.

Of course, but that is a small subset of the arches mentioned in
MACHINES, and anyway all windows, linux and solaris users assume that
Wget will keep on running on those systems. That is, MACHINE will
keep on having outdated or useless information.

Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://raul.pleyades.net/