Re: [Bug-wget] [PATCH v2 01/27] new Metalink functionalities

2016-10-03 Thread Matthew White
On Mon, 03 Oct 2016 13:56:39 +0200 Giuseppe Scrivano wrote: > Tim Ruehsen writes: > > > @Giuseppe Please go ahead with merging. > > I've just pushed the series. Thanks guys! > > Regards, > Giuseppe Regards, Matthew -- Matthew White pgptF6OjBmjXi.pgp Description: PGP signature

Re: [Bug-wget] [PATCH 21/27] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-30 Thread Matthew White
Hi, On Thu, 29 Sep 2016 06:03:01 +0200 Matthew White wrote: > + while ((n = FILE_SYSTEM_PREFIX_LEN (p))) > +p += n; Fix attached to silence Coverity's false positive. Thanks to Tim Ruehsen and Darshit Shah: http://lists.gnu.org/archive/html/bug-wget/2016-09/msg0017

Re: [Bug-wget] [PATCH 11/27] Enforce Metalink file name verification, strip directory if necessary

2016-09-30 Thread Matthew White
Hi, On Thu, 29 Sep 2016 06:02:51 +0200 Matthew White wrote: > + while ((n = FILE_SYSTEM_PREFIX_LEN (basename))) > +basename += n; Fix attached to silence Coverity's false positive. Thanks to Tim Ruehsen and Darshit Shah: http://lists.gnu.org/archive/html/bug-wget/2016-09/ms

Re: [Bug-wget] [PATCH v2 01/27] new Metalink functionalities

2016-09-30 Thread Matthew White
Hi Tim, On Fri, 30 Sep 2016 12:01:49 +0200 Tim Ruehsen wrote: > On Thursday, September 29, 2016 10:46:52 PM CEST Giuseppe Scrivano wrote: > > Hi Matthew, > > > > Matthew White writes: > > > Series of patches to implement new Metalink functionalities. > > @

Re: [Bug-wget] [PATCH 21/27] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-29 Thread Matthew White
Hi Giuseppe, On Thu, 29 Sep 2016 22:31:21 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > > diff --git a/src/init.c b/src/init.c > > index 6729c5a..26f3886 100644 > > --- a/src/init.c > > +++ b/src/init.c > &g

Re: [Bug-wget] [PATCH v2 01/27] new Metalink functionalities

2016-09-29 Thread Matthew White
Hi Giuseppe, On Thu, 29 Sep 2016 22:46:52 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > > Series of patches to implement new Metalink functionalities. > > > > In response to Giuseppe > > http://lists.gnu.org/archive/html/bug-w

Re: [Bug-wget] [PATCH 25/25] New: --metalink-over-http Content-Type/Disposition Metalink/XML processing

2016-09-29 Thread Matthew White
Hi Giuseppe, On Mon, 26 Sep 2016 10:23:31 +0200 Giuseppe Scrivano wrote: > Hi, > > Matthew White writes: > > >> Could you please re-send the updated series? > > > > You may find the series of patches in my branch > > https://github.com/mehw/wget/tree/

[Bug-wget] [PATCH 17/27] Bugfix: Remove surrounding quotes from Metalink/HTTP key's value

2016-09-28 Thread Matthew White
* src/metalink.h: Add declaration of function dequote_metalink_string() * src/metalink.c: Add function dequote_metalink_string() remove surrounding quotes from string, \' or \" * src/metalink.c (find_key_value, find_key_values): Call dequote_metalink_string() to remove the surrounding quotes f

[Bug-wget] [PATCH 16/27] Bugfix: Process Metalink/XML url strings containing white spaces and CRLF

2016-09-28 Thread Matthew White
* src/metalink.h: Add declaration of function clean_metalink_string() * src/metalink.c: Add directive #include "xmemdup0.h" * src/metalink.c: Add function clean_metalink_string() remove leading and trailing white spaces and CRLF from string * src/metalink.c (retrieve_from_metalink): Remove leadin

[Bug-wget] [PATCH 10/27] Implement Metalink/XML --directory-prefix option in Metalink module

2016-09-28 Thread Matthew White
* NEWS: Mention the effect of --directory-prefix over Metalink * src/metalink.c (retrieve_from_metalink): Add opt.dir_prefix as prefix to the metalink:file name mfile->name * doc/metalink.txt: Update document. Explain --directory-prefix When --directory-prefix= is used, set the top of the retrie

[Bug-wget] [PATCH 15/27] New test: Detect when there are no good Metalink url resources

2016-09-28 Thread Matthew White
* testenv/Makefile.am: Add new file * testenv/Test-metalink-xml-nourls.py: New file. Metalink/XML unknown urls tests Test if when there are no good Metalink url resources there is any segmentation fault. --- testenv/Makefile.am | 3 +- testenv/Test-metalink-xml-nourls.py | 195

[Bug-wget] [PATCH 26/27] New: --metalink-over-http Content-Type/Disposition Metalink/XML processing

2016-09-28 Thread Matthew White
* src/http.c (metalink_from_http): Process the Content-Type header. Add an application/metalink4+xml URL as metalink metaurl. If the option opt.content_disposition is true, the Content-Disposition's filename is the metaurl's name * doc/wget.texi: Update --content-disposition and --metalink-o

[Bug-wget] [PATCH 08/27] Add file size computation in Metalink module

2016-09-28 Thread Matthew White
* NEWS: Mention Metalink's file size verification * src/metalink.c (retrieve_from_metalink): Add file size computation * doc/metalink.txt: Update document. Remove resolved bugs Reject downloaded files when they do not agree with their Metalink/XML metalink:size: https://tools.ietf.org/html/rfc5854

[Bug-wget] [PATCH 25/27] Bugfix: Set NULL variable due to --content-disposition to Metalink origin

2016-09-28 Thread Matthew White
* src/http.c (http_loop): Prevent SIGSEGV when hstat.local_file is NULL, opt.content_disposition has a role in leaving the value unset * src/http.c (gethttp): If hs->local_file is NULL (aka http_loop()'s hstat.local_file), set it to the value of hs->metalink->origin --- src/http.c | 8 +++-

[Bug-wget] [PATCH 24/27] New: --trust-server-names saves Metalink/HTTP xml files using the "name" field

2016-09-28 Thread Matthew White
* src/metalink.c (retrieve_from_metalink): If opt.trustservernames is true, use the basename of the metaurl's name to save the xml file * doc/metalink-standard.txt: Update doc. With --trust-server-names any Metalink/HTTP Link application/metalink4+xml file is saved using the basename of the "

[Bug-wget] [PATCH 23/27] Bugfix: Detect when a metalink:file doesn't have any hash

2016-09-28 Thread Matthew White
* src/metalink.c (retrieve_from_metalink): Reject any metalink:file without hashes. Prompt the error and switch to the next file * testenv/Makefile.am: Add new file * testenv/Test-metalink-xml-nohash.py: New file. Metalink/XML with no hashes tests Prevent SIGSEGV. --- src/metalink.c

[Bug-wget] [PATCH 13/27] New: Metalink/XML and Metalink/HTTP file naming safety rules

2016-09-28 Thread Matthew White
* NEWS: Mention the effect of --trust-server-names over Metalink * src/metalink.h: Add declaration of function append_suffix_number() * src/metalink.c: Add function append_suffix_number() append number to string * src/metalink.c (retrieve_from_metalink): Safer Metalink/XML and Metalink/HTTP dow

[Bug-wget] [PATCH 09/27] Change mfile->name to filename in Metalink module's messages

2016-09-28 Thread Matthew White
* src/metalink.c (retrieve_from_metalink): Change mfile->name to filename when referring to the downloaded file The file name could have been changed by unique_create() (or by any other mean) before downloading. Use the name of the downloaded file (filename) when printing output which refer to i

[Bug-wget] [PATCH 21/27] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-28 Thread Matthew White
* NEWS: Mention the effect of --metalink-index over Metalink * src/init.c: Add new option metalinkindex (opt.metalink_index), initialize to -1 * src/main.c: Add new option metalink-index (--metalink-index=NUMBER) * src/options.h: Add new option metalink_index (int) * src/metalink.h: Add declarati

[Bug-wget] [PATCH 07/27] Update Metalink/XML tests and add a new test for home paths

2016-09-28 Thread Matthew White
* testenv/Test-metalink-xml-relpath.py: Update test * testenv/Test-metalink-xml-homepath.py: New file. Reject home paths * testenv/Makefile.am: Add new file When --input-metalink= is used, each metalink:file name is verified by libmetalink's metalink_check_safe_path(). By design, absolute, relativ

[Bug-wget] [PATCH 22/27] Bugfix: Detect malformed base64 Metalink/HTTP Digest header

2016-09-28 Thread Matthew White
* src/http.c (metalink_from_http): Fix hash_bin_len type. Use ssize_t instead than size_t. Reject -1 as base64_decode() return value * testenv/Makefile.am: Add new file * testenv/Test-metalink-http-baddigest.py: New file. Metalink/HTTP malformed base64 Digest header tests On malformed base64 i

[Bug-wget] [PATCH 06/27] Bugfix: Keep the download progress when alternating metalink:url

2016-09-28 Thread Matthew White
* NEWS: Mention the effects of --continue over Metalink * src/metalink.c (retrieve_from_metalink): On download error, resume output_stream with the next mres->url. Keep fully downloaded files started with --continue, otherwise rename/remove the file * testenv/Makefile.am: Add new file * testenv

[Bug-wget] [PATCH 20/27] Bugfix: Prevent sorting when there are less than two elements

2016-09-28 Thread Matthew White
* src/utils.c (stable_sort): Add condition nmemb > 1, sort only when there is more than one element Prevent SIGSEGV. --- src/utils.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/utils.c b/src/utils.c index 9ab1b90..dcf90a6 100644 --- a/src/utils.c +++ b/src/utils.c @@

[Bug-wget] [PATCH 19/27] New: Parse Metalink/HTTP header for application/metalink4+xml

2016-09-28 Thread Matthew White
* src/http.c (metalink_from_http): Parse Metalink/HTTP header for metaurls application/metalink4+xml media types * src/metalink.h: Add function declaration metalink_meta_cmp() * src/metalink.c: Add function metalink_meta_cmp() compare metalink metaurls priorities Add Metalink/HTTP application/

[Bug-wget] [PATCH 18/27] New test: Metalink shall not concatenate '/' to an empty directory prefix

2016-09-28 Thread Matthew White
* testenv/Makefile.am: Add new file * testenv/Test-metalink-xml-emptyprefix-trust.py: New file. Metalink/XML empty directory prefix (--directory-prefix '') tests Detect a '/' character wrongfully concatenated to an empty directory prefix '' (not NULL), resulting in an absolute path as '/dir/file

[Bug-wget] [PATCH 03/27] Use python .replace instead than re.sub in Metalink tests

2016-09-28 Thread Matthew White
* testenv/Test-metalink-http.py: Use python .replace * testenv/Test-metalink-xml.py: Use python .replace * testenv/Test-metalink-xml-abspath.py: Use python .replace * testenv/Test-metalink-xml-relpath.py: Use python .replace Use python .replace instead than re.sub, remove 'import re'. --- testenv

[Bug-wget] [PATCH 14/27] New: Metalink file size mismatch returns error code METALINK_SIZE_ERROR

2016-09-28 Thread Matthew White
* src/wget.h (uerr_t): Add error code METALINK_SIZE_ERROR to enum * src/metalink.c (retrieve_from_metalink): Use boolean variable size_ok, when false set retr_err to METALINK_SIZE_ERROR * testenv/Makefile.am: Add new file * testenv/Test-metalink-xml-size.py: New file. Metalink/XML file size tes

[Bug-wget] [PATCH 05/27] Bugfix: Fix NULL filename and output_stream in Metalink module

2016-09-28 Thread Matthew White
* NEWS: Mention the Metalink "path/file" name format handling * src/metalink.c (retrieve_from_metalink): Fix NULL filename, set filename to the right "path/file" value * src/metalink.c (retrieve_from_metalink): Fix NULL output_stream, set output_stream to filename when it is created by retrieve

[Bug-wget] [PATCH 04/27] Add metalink description

2016-09-28 Thread Matthew White
* doc/metalink.txt Evaluation of "Directory Options" on the command line interacting with the option '--input-metalink=file': $ wget --input-metalink=file --- doc/metalink.txt | 137 +++ 1 file changed, 137 insertions(+) create mode 100644 do

[Bug-wget] [PATCH 12/27] New document: Metalink/XML and Metalink/HTTP standard reference

2016-09-28 Thread Matthew White
* doc/metalink-standard.txt: New doc. Implemented and recommended Metalink/XML and Metalink/HTTP standard features --- doc/metalink-standard.txt | 156 ++ 1 file changed, 156 insertions(+) create mode 100644 doc/metalink-standard.txt diff --git a/doc

[Bug-wget] [PATCH 11/27] Enforce Metalink file name verification, strip directory if necessary

2016-09-28 Thread Matthew White
* NEWS: Mention the use of a safe Metalink destination path * src/metalink.h: Add declaration of functions get_metalink_basename(), last_component(), metalink_check_safe_path() * src/metalink.c: Add directive #include "dosname.h" * src/metalink.c: Add function get_metalink_basename() to return th

[Bug-wget] [PATCH 01/27] Add two Metalink/XML tests

2016-09-28 Thread Matthew White
From: Tim Rühsen * testenv/Test-metalink-xml-abspath.py: Reject absolute paths * testenv/Test-metalink-xml-relpath.py: Reject relative paths * testenv/Makefile.am: Add both new files to metalink tests --- testenv/Makefile.am | 6 ++- testenv/Test-metalink-xml-abspath.py | 87 ++

[Bug-wget] [PATCH 02/27] Fix: Change Metalink/XML v3 file name into test.metalink

2016-09-28 Thread Matthew White
* testenv/Test-metalink-xml-abspath.py: Change Metalink/XML v3 file name from test.meta4 into test.metalink * testenv/Test-metalink-xml-relpath.py: Change Metalink/XML v3 file name from test.meta4 into test.metalink * testenv/Test-metalink-xml.py: Change Metalink/XML v3 file name from test.me

[Bug-wget] [PATCH v2 01/27] new Metalink functionalities

2016-09-28 Thread Matthew White
Series of patches to implement new Metalink functionalities. In response to Giuseppe http://lists.gnu.org/archive/html/bug-wget/2016-09/msg00127.html , here you find my revised series of patches. Posting with `git send-email` as requested. This series of patches supersedes the following: [Pat

Re: [Bug-wget] [PATCH 25/25] New: --metalink-over-http Content-Type/Disposition Metalink/XML processing

2016-09-25 Thread Matthew White
Hi Giuseppe, On Sun, 25 Sep 2016 19:52:18 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > > [Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, > > contrib/check-hard is ok] > > > > This implements the au

Re: [Bug-wget] [PATCH 20/25] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-16 Thread Matthew White
On Fri, 16 Sep 2016 10:15:17 +0200 Tim Ruehsen wrote: > On Thursday, September 15, 2016 7:56:45 PM CEST Matthew White wrote: > > On Thu, 15 Sep 2016 16:48:01 +0200 > > > > Tim Ruehsen wrote: > > > On Thursday, September 15, 2016 4:16:44 PM CEST Matthew White wrot

Re: [Bug-wget] [PATCH 20/25] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-15 Thread Matthew White
On Thu, 15 Sep 2016 16:48:01 +0200 Tim Ruehsen wrote: > On Thursday, September 15, 2016 4:16:44 PM CEST Matthew White wrote: > > On Thu, 15 Sep 2016 09:11:31 +0200 > > > > Giuseppe Scrivano wrote: > > > Hi Matthew, > > > > > > Matthew White wri

Re: [Bug-wget] [PATCH 20/25] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-15 Thread Matthew White
On Thu, 15 Sep 2016 09:11:31 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > >> > Function amended to modify *name in place. > >> > > >> > Followed Tim's suggestions for Patch 09/25 about different envi

Re: [Bug-wget] [PATCH 20/25] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-14 Thread Matthew White
On Wed, 14 Sep 2016 13:03:58 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > > On Sun, 11 Sep 2016 23:39:09 +0200 > > Giuseppe Scrivano wrote: > > > >> Hi Matthew, > >> > >> Matthew White writes: > >

Re: [Bug-wget] [PATCH 20/25] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-14 Thread Matthew White
On Sun, 11 Sep 2016 23:39:09 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > > +void > > +replace_metalink_basename (char **name, char *ref) > > +{ > > + size_t dir_len = 0; > > + char *p, *dir, *file, *ne

Re: [Bug-wget] [PATCH 09/25] Enforce Metalink file name verification, strip directory if necessary

2016-09-13 Thread Matthew White
On Tue, 13 Sep 2016 16:21:37 +0200 Tim Ruehsen wrote: > On Tuesday, September 13, 2016 4:08:30 PM CEST Matthew White wrote: > > On Tue, 13 Sep 2016 09:29:27 +0200 > > > > Tim Ruehsen wrote: > > > On Tuesday, September 13, 2016 5:13:10 AM CEST Matthew White wrote:

Re: [Bug-wget] [PATCH 09/25] Enforce Metalink file name verification, strip directory if necessary

2016-09-13 Thread Matthew White
On Tue, 13 Sep 2016 09:29:27 +0200 Tim Ruehsen wrote: > On Tuesday, September 13, 2016 5:13:10 AM CEST Matthew White wrote: > > On Mon, 12 Sep 2016 21:20:54 +0200 > > > > Tim Rühsen wrote: > > > On Montag, 12. September 2016 20:18:30 CEST Eli Zaretskii wrot

Re: [Bug-wget] [PATCH 04/25] Bugfix: Keep the download progress when alternating metalink:url

2016-09-13 Thread Matthew White
On Sun, 11 Sep 2016 22:04:41 +0200 Giuseppe Scrivano wrote: > Matthew White writes: > > > From f9fc03c0788675275041d0876d3e7fffd3f50eee Mon Sep 17 00:00:00 2001 > > From: Matthew White > > Date: Thu, 4 Aug 2016 11:35:42 +0200 > > Subject: [PATCH 04/25] Bugfix: K

Re: [Bug-wget] [PATCH 12/25] New test: --continue shall keep fully retrieved Metalink files (HTTP 416)

2016-09-13 Thread Matthew White
On Sun, 11 Sep 2016 22:51:27 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > > [Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, > > contrib/check-hard is ok] > > > > This introduces a new Metalink test to

Re: [Bug-wget] [PATCH 01/25] Add two Metalink/XML tests

2016-09-13 Thread Matthew White
On Tue, 13 Sep 2016 10:10:49 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > > Next patches add more conditions to testenv/Test-metalink-xml-relpath.py. > > > > Also variants are introduced, like: > > * testenv/Test-metalink-xml-

Re: [Bug-wget] [PATCH 05/25] Update Metalink/XML tests and add a new test for home paths

2016-09-13 Thread Matthew White
On Sun, 11 Sep 2016 22:06:50 +0200 Giuseppe Scrivano wrote: > Matthew White writes: > > > + > > same as before, can be just another test in the same file, not a > complete new test file. Metalink tests are specialized and may be revisioned later on adding more c

Re: [Bug-wget] [PATCH 10/25] New document: Metalink/XML and Metalink/HTTP standard reference

2016-09-13 Thread Matthew White
On Sun, 11 Sep 2016 22:45:09 +0200 Giuseppe Scrivano wrote: > Matthew White writes: > > > [Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, > > contrib/check-hard is ok] > > > > This introduces the new document doc/metalink-standard.t

Re: [Bug-wget] [PATCH 16/25] Bugfix: Remove surrounding quotes from Metalink/HTTP key's value

2016-09-13 Thread Matthew White
On Sun, 11 Sep 2016 23:10:55 +0200 Giuseppe Scrivano wrote: > Matthew White writes: > > > +/* > > + Remove the quotation surrounding a string. > > + > > + The string is permanently modified. > > + */ > > +void > > +dequote_metalink_string

Re: [Bug-wget] [PATCH 17/25] New test: Metalink shall not concatenate '/' to an empty directory prefix

2016-09-12 Thread Matthew White
On Sun, 11 Sep 2016 23:16:56 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > Matthew White writes: > > > Matthew White > > > > From 9f1492729e3771405b13b9375be45e76fdffad62 Mon Sep 17 00:00:00 2001 > > From: Matthew White > > Date: Sat, 27 Aug 201

Re: [Bug-wget] [PATCH 19/25] Bugfix: Prevent sorting of unallocated metalink resource/metaurl

2016-09-12 Thread Matthew White
On Sun, 11 Sep 2016 23:27:10 +0200 Giuseppe Scrivano wrote: > Matthew White writes: > > > diff --git a/src/main.c b/src/main.c > > index ac6ee2c..11ea86d 100644 > > --- a/src/main.c > > +++ b/src/main.c > > @@ -2136,10 +2136,11 @@ onl

Re: [Bug-wget] [PATCH 14/25] New test: Detect when there are no good Metalink url resources

2016-09-12 Thread Matthew White
, str (srv_port), MetaXml) > > +MetaFile.content = MetaXml > > and here again could we skip the re module? ACK with it changed. Thanks, fixed here and in other patches. Posting after final decisions are taken about open topics in this series of patches. > > Giuseppe Regards, Matthew -- Matthew White pgpO0zCUTV48i.pgp Description: PGP signature

Re: [Bug-wget] [PATCH 08/25] Implement Metalink/XML --directory-prefix option in Metalink module

2016-09-12 Thread Matthew White
fix && strlen (opt.dir_prefix)) > > +filename = aprintf("%s/%s", opt.dir_prefix, mfile->name); > > please leave a space between aprintf and '('. Thanks, fixed here and in other patches. Posting after final decisions are taken about open topics in this

Re: [Bug-wget] [PATCH 03/25] Bugfix: Fix NULL filename and output_stream in Metalink module

2016-09-12 Thread Matthew White
On Sun, 11 Sep 2016 21:59:12 +0200 Giuseppe Scrivano wrote: > Matthew White writes: > > > + */ > > + if (!output_stream && (output_stream = fopen (filename, > > "rb"))) > > +

Re: [Bug-wget] [PATCH 01/25] Add two Metalink/XML tests

2016-09-12 Thread Matthew White
On Sun, 11 Sep 2016 21:52:16 +0200 Giuseppe Scrivano wrote: > Hi Matthew, > > thanks for all your work, I am going to comment on each patch separately: > > Matthew White writes: > > > > index 000..041d772 > > --- /dev/null > > +++ b/testenv/Test-me

Re: [Bug-wget] [PATCH 09/25] Enforce Metalink file name verification, strip directory if necessary

2016-09-12 Thread Matthew White
* get_metalink_basename (char *name) { char *basename; if (!name) return NULL; basename = name + strlen (name); while (basename > name && !ISSEP (*basename)) --basename; if (ISSEP (*basename)) ++basename; return metalink_check_safe_path (basename) ? basename : NULL; } Regards, Matthew -- Matthew White pgpk3G7_fjcQY.pgp Description: PGP signature

Re: [Bug-wget] [PATCH 09/25] Enforce Metalink file name verification, strip directory if necessary

2016-09-12 Thread Matthew White
On Mon, 12 Sep 2016 13:00:32 +0200 Tim Ruehsen wrote: > On Saturday, September 10, 2016 12:49:50 PM CEST Matthew White wrote: > > > +int metalink_check_safe_path(const char *path); > > One space before (, please ;-) Fixed. Posting after final decisions are taken about o

Re: [Bug-wget] [PATCH 11/25] New: Metalink/XML and Metalink/HTTP file naming safety rules

2016-09-12 Thread Matthew White
On Sun, 11 Sep 2016 22:48:55 +0200 Giuseppe Scrivano wrote: > Matthew White writes: > > > [Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, > > contrib/check-hard is ok] > > > > This introduces new rules/tests about Metalink/XML and Met

Re: [Bug-wget] [PATCH 09/25] Enforce Metalink file name verification, strip directory if necessary

2016-09-12 Thread Matthew White
/* On systems with drive letters, "a/b:c" must return "./b:c" rather than "b:c" to avoid confusion with a drive letter. On systems with pure POSIX semantics, this is not an issue. */ -- Suggestions? > > Regards, Tim Regards, Matthew -- Matthew White pgpC6jURyKvHd.pgp Description: PGP signature

[Bug-wget] [PATCH 25/25] New: --metalink-over-http Content-Type/Disposition Metalink/XML processing

2016-09-10 Thread Matthew White
trust-server-names is also required. ----- Regards, Matthew -- Matthew White >From ec75cdd3f0e748523c64856f01b28128f806fb16 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Mon, 29 Aug 2016 20:59:35 +0200 Subject: [PATCH 25/25] New: --metalink-over-http Content-Type/Disposition

[Bug-wget] [PATCH 22/25] Bugfix: Detect when a metalink:file doesn't have any hash

2016-09-10 Thread Matthew White
esn't provide any hash, preventing a segmentation fault. Regards, Matthew -- Matthew White >From 3555fe04ab1fc22c93d48f88584c7116d0356bb5 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Sun, 28 Aug 2016 14:03:56 +0200 Subject: [PATCH 22/25] Bugfix: Detect when a metalink:file doesn

[Bug-wget] [PATCH 24/25] Bugfix: Set NULL variable due to --content-disposition to Metalink origin

2016-09-10 Thread Matthew White
local_file. Regards, Matthew -- Matthew White >From 68e71eba7b3384543c46f43e873421ee0b8722a0 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Tue, 30 Aug 2016 12:09:38 +0200 Subject: [PATCH 24/25] Bugfix: Set NULL variable due to --content-disposition to Metalink origin * src/htt

[Bug-wget] [PATCH 21/25] Bugfix: Detect malformed base64 Metalink/HTTP Digest header

2016-09-10 Thread Matthew White
Digest header. The following description is verbatim from the patch: - On malformed base64 input, ssize_t base64_decode() returns -1. Such value is too big for a size_t variable, and used as xmalloc() value will exaust all the memory. - Regards, Matthew -- Matthew White >F

[Bug-wget] [PATCH 23/25] New: --trust-server-names saves Metalink/HTTP xml files using the "name" field

2016-09-10 Thread Matthew White
ver-names is used, file.ext.meta4 is saved as newname.meta4. Regards, Matthew -- Matthew White >From 30b09eb7dbfbb9423efb6bd4009203f887d90786 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Tue, 30 Aug 2016 09:26:41 +0200 Subject: [PATCH 23/25] New: --trust-server-names saves Metalink/HTTP x

[Bug-wget] [PATCH 20/25] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-10 Thread Matthew White
nf means the first good one. Regards, Matthew -- Matthew White >From bb43db44f229b2fcd492860ffa9b73ee407f691b Mon Sep 17 00:00:00 2001 From: Matthew White Date: Fri, 26 Aug 2016 11:19:24 +0200 Subject: [PATCH 20/25] New option --metalink-index to process Metalink application/metalink4+xml *

[Bug-wget] [PATCH 19/25] Bugfix: Prevent sorting of unallocated metalink resource/metaurl

2016-09-10 Thread Matthew White
. - Regards, Matthew -- Matthew White >From cbed79d8394c8c0a74b2f65cb92d01178184e5fe Mon Sep 17 00:00:00 2001 From: Matthew White Date: Thu, 25 Aug 2016 15:26:18 +0200 Subject: [PATCH 19/25] Bugfix: Prevent sorting of unallocated metalink resource/metaurl * src/main.c (main): Sort metalink resour

[Bug-wget] [PATCH 18/25] New: Parse Metalink/HTTP header for application/metalink4+xml

2016-09-10 Thread Matthew White
the files. - Regards, Matthew -- Matthew White >From e1ff59ba9086bac0211e2cb54ef2b91038b45590 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Thu, 25 Aug 2016 05:09:47 +0200 Subject: [PATCH 18/25] New: Parse Metalink/HTTP header for application/metalink4+xml * src/http.c (metal

[Bug-wget] [PATCH 14/25] New test: Detect when there are no good Metalink url resources

2016-09-10 Thread Matthew White
, Matthew -- Matthew White >From fe1c53b831387742e0d21c5e0fd1935596f4b1ed Mon Sep 17 00:00:00 2001 From: Matthew White Date: Tue, 23 Aug 2016 04:57:33 +0200 Subject: [PATCH 14/25] New test: Detect when there are no good Metalink url resources * testenv/Makefile.am: Add new file * testenv/T

[Bug-wget] [PATCH 15/25] Bugfix: Process Metalink/XML url strings containing white spaces and CRLF

2016-09-10 Thread Matthew White
tains strings separated by CRLF, only the first of the series is accepted. - Regards, Matthew -- Matthew White >From d213fedd77668d6be891b9be4cd8de4b077a2c49 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Tue, 23 Aug 2016 21:59:15 +0200 Subject: [PATCH 15/25] Bugfix: Process Metalink

[Bug-wget] [PATCH 17/25] New test: Metalink shall not concatenate '/' to an empty directory prefix

2016-09-10 Thread Matthew White
NULL), resulting in an absolute path as '/dir/file', instead than 'dir/file'. ----- Regards, Matthew -- Matthew White >From 9f1492729e3771405b13b9375be45e76fdffad62 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Sat, 27 Aug 2016 16:16:35 +0200 Subject: [PATCH 17/2

[Bug-wget] [PATCH 16/25] Bugfix: Remove surrounding quotes from Metalink/HTTP key's value

2016-09-10 Thread Matthew White
d Hashes 1.1. Example Metalink Server Response https://tools.ietf.org/html/rfc6249#section-1.1 [2] Additional Link Relations 6. "type" https://tools.ietf.org/html/rfc6903#section-6 - Regards, Matthew -- Matthew White >From b062d348e50ac1a7789e1f322de9981

[Bug-wget] [PATCH 12/25] New test: --continue shall keep fully retrieved Metalink files (HTTP 416)

2016-09-10 Thread Matthew White
batim from the patch: - Ensure that --continue doesn't rename/remove existing and/or fully retrieved files which fail the sanity tests. - Regards, Matthew -- Matthew White >From 9b178b179ba570a201e1df444ee7f310eea00bfd Mon Sep 17 00:00:00 2001 From: Matthew White Date: Mon, 22 A

[Bug-wget] [PATCH 13/25] New: Metalink file size mismatch returns error code METALINK_SIZE_ERROR

2016-09-10 Thread Matthew White
was returned to inform a file size mismatch. This patch introduces the error code METALINK_SIZE_ERROR to inform a file size mismatch. - Regards, Matthew -- Matthew White >From 448e3c06cd9589509cd3914a43fe4c7e93e9d622 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Mon, 22 Aug 2016 09

[Bug-wget] [PATCH 11/25] New: Metalink/XML and Metalink/HTTP file naming safety rules

2016-09-10 Thread Matthew White
used, a suffix could be appended to the final name to not overwrite existing files. Regards, Matthew -- Matthew White >From 7fef56c4f815359047347eee4356b9f8971f1df5 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Sun, 21 Aug 2016 18:45:09 +0200 Sub

[Bug-wget] [PATCH 10/25] New document: Metalink/XML and Metalink/HTTP standard reference

2016-09-10 Thread Matthew White
currently implemented in Wget. Regards, Matthew -- Matthew White >From 40442c885ab06dbef19caeef6bc4ba22a26dbb31 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Fri, 19 Aug 2016 13:17:34 +0200 Subject: [PATCH 10/25] New document: Metalink/XML and Metalink/HTTP standard reference * doc/metal

[Bug-wget] [PATCH 09/25] Enforce Metalink file name verification, strip directory if necessary

2016-09-10 Thread Matthew White
of unsafe file names. The following description is verbatim from the patch: - Unsafe file names contain an absolute, relative, or home path. Safe paths can be verified by libmetalink's metalink_check_safe_path(). - Regards, Matthew -- Matth

[Bug-wget] [PATCH 08/25] Implement Metalink/XML --directory-prefix option in Metalink module

2016-09-10 Thread Matthew White
retrieval tree to prefix. The default is . (the current directory). Metalink/XML and Metalink/HTTP files will be downloaded under prefix. - Regards, Matthew -- Matthew White >From 43840eac31697a06e15924976d25b91c697942f6 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Wed, 17 Aug 2016

[Bug-wget] [PATCH 07/25] Change mfile->name to filename in Metalink module's messages

2016-09-10 Thread Matthew White
by any other mean) before downloading. Use the name of the downloaded file (filename) when printing output which refer to it. - Regards, Matthew -- Matthew White >From 0a41d7a593bf8b1be7231146af2b8bdf7755e4fa Mon Sep 17 00:00:00 2001 From: Matthew White Date: Tue, 16 Aug 2016 23:18:46 +0

[Bug-wget] [PATCH 06/25] Add file size computation in Metalink module

2016-09-10 Thread Matthew White
^ wget.h:130:54: note: in definition of macro 'DEBUGP' #define DEBUGP(args) do { IF_DEBUG { debug_logprintf args; } } while (0) ^ Regards, Matthew -- Matthew White >From f9da9729d70d79ce245877672a2d611ca9c4667f Mon Sep 17 00:

[Bug-wget] [PATCH 04/25] Bugfix: Keep the download progress when alternating metalink:url

2016-09-10 Thread Matthew White
doesn't rename/remove existing and/or fully retrieved files which fail the sanity tests. - Regards, Matthew -- Matthew White >From f9fc03c0788675275041d0876d3e7fffd3f50eee Mon Sep 17 00:00:00 2001 From: Matthew White Date: Thu, 4 Aug 2016 11:35:42 +0200 Subject: [PATCH 04/25] Bugfix:

[Bug-wget] [PATCH 05/25] Update Metalink/XML tests and add a new test for home paths

2016-09-10 Thread Matthew White
ative, and home paths aren't a concern. The destination file name is a combination of URL's file name and cli's "Directory Options" handled by src/url.c (url_file_name). - Regards, Matthew -- Matthew White >From 9812e43397d3bdb3b1488a635558f89078e4a945 Mon Sep 17

[Bug-wget] [PATCH 02/25] Add metalink description

2016-09-10 Thread Matthew White
. Regards, Matthew -- Matthew White >From 2a418049a0678f781ff03cecd4bde4ecfdffec2e Mon Sep 17 00:00:00 2001 From: Matthew White Date: Fri, 5 Aug 2016 20:10:19 +0200 Subject: [PATCH 02/25] Add metalink description * doc/metalink.txt Evaluation of "Directory Options" on the command lin

[Bug-wget] [PATCH 03/25] Bugfix: Fix NULL filename and output_stream in Metalink module

2016-09-10 Thread Matthew White
rary. - Regards, Matthew -- Matthew White >From e3e69bc551a0f3794f8da3fdb421c2868d6e8326 Mon Sep 17 00:00:00 2001 From: Matthew White Date: Thu, 28 Jul 2016 17:10:46 +0200 Subject: [PATCH 03/25] Bugfix: Fix NULL filename and output_stream in Metalink module * src/metalink.c (retrieve_f

[Bug-wget] [PATCH 01/25] Add two Metalink/XML tests

2016-09-10 Thread Matthew White
-4.1.2.1 This adds two new Metalink tests to testenv: * testenv/Test-metalink-xml-relpath.py: Test if Metalink/XML forbids relative paths * testenv/Test-metalink-xml-abspath.py: Test if Metalink/XML forbids absolute paths Regards, Matthew -- Matthew White >F

Re: [Bug-wget] [PATCH] Implement --keep-badhash to keep Metalink's files with a bad hash

2016-08-19 Thread Matthew White
On Fri, 19 Aug 2016 12:16:15 +0200 Tim Rühsen wrote: > On Freitag, 19. August 2016 07:34:42 CEST Matthew White wrote: > > On Thu, 18 Aug 2016 12:50:48 +0200 > > > > Tim Rühsen wrote: > > > On Mittwoch, 17. August 2016 17:55:47 CEST Matthew White wrote: > &

Re: [Bug-wget] [PATCH] Implement --keep-badhash to keep Metalink's files with a bad hash

2016-08-18 Thread Matthew White
On Thu, 18 Aug 2016 12:50:48 +0200 Tim Rühsen wrote: > On Mittwoch, 17. August 2016 17:55:47 CEST Matthew White wrote: > > On Thu, 04 Aug 2016 17:08:20 +0200 > > > > Tim Ruehsen wrote: > > > On Thursday, August 4, 2016 12:40:49 PM CEST Matthew White wrote: > &

Re: [Bug-wget] [PATCH] Implement --keep-badhash to keep Metalink's files with a bad hash

2016-08-17 Thread Matthew White
On Thu, 04 Aug 2016 17:08:20 +0200 Tim Ruehsen wrote: > On Thursday, August 4, 2016 12:40:49 PM CEST Matthew White wrote: > > On Thu, 04 Aug 2016 12:08:50 +0200 > > > > Tim Ruehsen wrote: > > > On Wednesday, August 3, 2016 2:40:14 PM CEST Matthew White wrote: >

Re: [Bug-wget] Wget tests

2016-08-15 Thread Matthew White
e] $ make check Good luck, Matthew -- Matthew White pgpTxqF2Uyga3.pgp Description: PGP signature

Re: [Bug-wget] [PATCH] Support metalink:file elements with a "path/file" format

2016-08-14 Thread Matthew White
On Sat, 13 Aug 2016 11:57:46 +0200 Tim Rühsen wrote: > On Freitag, 12. August 2016 22:13:53 CEST Matthew White wrote: > > On Wed, 10 Aug 2016 11:30:12 +0200 > > > > After debugging wget and libmetalink, I can confirm that, due to how > > metalink/libmetalink i

Re: [Bug-wget] [PATCH] Support metalink:file elements with a "path/file" format

2016-08-12 Thread Matthew White
On Wed, 10 Aug 2016 11:30:12 +0200 Tim Rühsen wrote: > On Mittwoch, 10. August 2016 06:23:49 CEST Matthew White wrote: > > On Tue, 09 Aug 2016 21:25:16 +0200 > > > > Tim Rühsen wrote: > > > On Freitag, 5. August 2016 20:25:06 CEST Matthew White wrote: > > &g

Re: [Bug-wget] [PATCH] Support metalink:file elements with a "path/file" format

2016-08-11 Thread Matthew White
On Wed, 10 Aug 2016 11:30:12 +0200 Tim Rühsen wrote: > On Mittwoch, 10. August 2016 06:23:49 CEST Matthew White wrote: > > On Tue, 09 Aug 2016 21:25:16 +0200 > > > > Tim Rühsen wrote: > > > On Freitag, 5. August 2016 20:25:06 CEST Matthew White wrote: > > &g

Re: [Bug-wget] [PATCH] Support metalink:file elements with a "path/file" format

2016-08-09 Thread Matthew White
On Tue, 09 Aug 2016 21:25:16 +0200 Tim Rühsen wrote: > On Freitag, 5. August 2016 20:25:06 CEST Matthew White wrote: > > On Thu, 04 Aug 2016 16:47:18 +0200 > > > > Tim Ruehsen wrote: > > > On Wednesday, August 3, 2016 1:46:56 PM CEST Matthew White wrote: > &

Re: [Bug-wget] [PATCH] Support metalink:file elements with a "path/file" format

2016-08-05 Thread Matthew White
On Thu, 04 Aug 2016 16:47:18 +0200 Tim Ruehsen wrote: > On Wednesday, August 3, 2016 1:46:56 PM CEST Matthew White wrote: > > On Tue, 02 Aug 2016 11:27:08 +0200 > > > > Tim Ruehsen wrote: > > > On Tuesday, August 2, 2016 10:06:42 AM CEST Matthew White wrote: >

Re: [Bug-wget] [PATCH] Implement --keep-badhash to keep Metalink's files with a bad hash

2016-08-05 Thread Matthew White
On Fri, 05 Aug 2016 10:28:39 +0200 Tim Ruehsen wrote: > On Friday, August 5, 2016 9:58:44 AM CEST Matthew White wrote: > > > > I posted 0001 and 0002 before (see links above), I attach them here too. > > > > > > Did you see that I answered to 0001 (security c

Re: [Bug-wget] [PATCH] Implement --keep-badhash to keep Metalink's files with a bad hash

2016-08-05 Thread Matthew White
On Fri, 05 Aug 2016 09:35:34 +0200 Tim Ruehsen wrote: > On Thursday, August 4, 2016 8:03:55 PM CEST Matthew White wrote: > > On Thu, 04 Aug 2016 17:08:20 +0200 > > > > Tim Ruehsen wrote: > > > On Thursday, August 4, 2016 12:40:49 PM CEST Matthew White wrote: >

Re: [Bug-wget] [PATCH] Implement --keep-badhash to keep Metalink's files with a bad hash

2016-08-05 Thread Matthew White
On Thu, 04 Aug 2016 17:08:20 +0200 Tim Ruehsen wrote: > On Thursday, August 4, 2016 12:40:49 PM CEST Matthew White wrote: > > On Thu, 04 Aug 2016 12:08:50 +0200 > > > > Tim Ruehsen wrote: > > > On Wednesday, August 3, 2016 2:40:14 PM CEST Matthew White wrote: >

Re: [Bug-wget] [PATCH] Implement --keep-badhash to keep Metalink's files with a bad hash

2016-08-04 Thread Matthew White
On Thu, 04 Aug 2016 17:08:20 +0200 Tim Ruehsen wrote: > On Thursday, August 4, 2016 12:40:49 PM CEST Matthew White wrote: > > On Thu, 04 Aug 2016 12:08:50 +0200 > > > > Tim Ruehsen wrote: > > > On Wednesday, August 3, 2016 2:40:14 PM CEST Matthew White wrote: >

Re: [Bug-wget] [PATCH] Remove hyphens from command names (Was: Re: Hyphens in command name (init.c) / option data (main.c))

2016-08-04 Thread Matthew White
gt; > > A patch would be very appreciated. > > Since it will (well, I guess so) be a 'trivial' patch, no FSF copyright > > assignment is needed. So, just go ahead ! > > > > Regards, Tim > > > > Thanks Tim - attached is the patch. Let me know if I should instead send it > inline in a new email. > > Best regards, > Jeff That's a nice one Jeff! Regards, Matthew -- Matthew White pgp7hBuRhk7sR.pgp Description: PGP signature

Re: [Bug-wget] [PATCH] Implement --keep-badhash to keep Metalink's files with a bad hash

2016-08-04 Thread Matthew White
On Thu, 04 Aug 2016 17:08:20 +0200 Tim Ruehsen wrote: > On Thursday, August 4, 2016 12:40:49 PM CEST Matthew White wrote: > > On Thu, 04 Aug 2016 12:08:50 +0200 > > > > Tim Ruehsen wrote: > > > On Wednesday, August 3, 2016 2:40:14 PM CEST Matthew White wrote: >

Re: [Bug-wget] buildbot failure in OpenCSW Buildbot on wget-solaris10-i386

2016-08-04 Thread Matthew White
> > 162 > > > > Buildbot URL: https://buildfarm.opencsw.org/buildbot/ > > > > Buildslave for this Build: unstable10x > > > > Build Reason: scheduler > > Build Source Stamp: [branch master] 943a6d585ff4a0afd974ab2e4838c9d2a63cf21a > > Blamelist: Matthew W

Re: [Bug-wget] buildbot failure in OpenCSW Buildbot on wget-solaris10-i386

2016-08-04 Thread Matthew White
On Thu, 04 Aug 2016 13:16:26 +0200 Tim Ruehsen wrote: > On Thursday, August 4, 2016 1:09:35 PM CEST Matthew White wrote: > > On Thu, 04 Aug 2016 12:17:36 +0200 > > > > build...@opencsw.org wrote: > > > The Buildbot has detected a new failure on builder wget-solar

  1   2   >