Re: [Bug-wget] Marking Release v1.17.1?

[Jernej Simončič]

On Tuesday, December 8, 2015, 21:45:31, Darshit Shah wrote:

> With my last set of patches, we have fixed all the issues reported / 
> identified
> after the 1.17 release. Hence, maybe we should consider releasing a 1.17.1
> bugfix release?

Here's another one that I thought was already fixed, but apparently
wasn't - --connect-timeout doesn't work on Windows without this patch
(it's not included in my 1.17 Win32 builds, and I got an e-mail about
this in the morning):

https://eternallybored.org/misc/wget/src/old/fix-timeout-win32.patch

-- 
< Jernej Simončič ><><><><>< http://eternallybored.org/ >

It is better to have a horrible ending than to have horrors without end.
   -- Matsch's Law

[Bug-wget] Fixing Test-k for Cygwin (and hopefully Windows)

[Tim Ruehsen]

Hi,

Test-k fails on Cygwin with LC_ALL=C (which is the default for 'make check').

With this patch the test works on CygWin and on Linux.

Here is a patch that *should* fix it also for 'plain' MS Windows.
Could anyone test this on Windows (without CygWin) ?

Regards, Tim
From 2c952b4d62492ee50ca7d2caa6edee4f0cfbd3e3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Tim Rühsen?= 
Date: Fri, 11 Dec 2015 11:43:40 +0100
Subject: [PATCH] Let Test-k survive on CygWin and Windows

* tests/Test-k.px: Use --restrict-file-names for CygWin/Windows
  filename requirements.
---
 tests/Test-k.px | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tests/Test-k.px b/tests/Test-k.px
index 9005c5f..ce114d7 100755
--- a/tests/Test-k.px
+++ b/tests/Test-k.px
@@ -7,6 +7,8 @@ use HTTPTest;


 ###
+my $osname = $^O;
+print "OS=$osname\n";

 my $index = <

Re: [Bug-wget] Windows cert store support

[Eli Zaretskii]

> Date: Thu, 10 Dec 2015 01:12:37 +0100
> From: Ángel González 
> Cc: bug-wget 
> 
> On 09/12/15 03:06, Random Coder wrote:
> > I'm not sure if the wget maintainers would be interested, but I've
> > been carrying this patch around in my private builds of wget for a
> > while.  It allows wget to load SSL certs from the default Windows cert
> > store.
> >
> > The patch itself is fairly straightforward, but as it changes the
> > default SSL behavior, and no care was taken to follow coding convents
> > when I wrote it, so it's probably not ready for inclusion in the
> > codebase.  Still, if it's useful, feel free to use it for ideas.
> Wow, supporting the OS store would certainly be very cool.
> 
> I would probably move it to windows.c and attempt to make it also work 
> in gnutls, but in general it looks good.

Wget compiled with GnuTLS already supports this feature: it calls
gnutls_certificate_set_x509_system_trust when the GnuTLS library
supports that.  gnutls_certificate_set_x509_system_trust does
internally what the proposed patch does.

So I think this code should indeed go only to openssl.c, as gnutls.c
already has its equivalent.

One other comment I have about the patch is that it's inconsistent
with what gnutls.c does:

  if (!opt.ca_directory)
ncerts = gnutls_certificate_set_x509_system_trust (credentials);
  /* If GnuTLS version is too old or CA loading failed, fallback to old 
behaviour.
   * Also use old behaviour if the CA directory is user-provided.  */
  if (ncerts <= 0)
{

IOW, condition the attempt to load the system certs on
opt.ca_directory, and fall back to the certs from files if that fails.

Thanks.

Re: [Bug-wget] Windows cert store support

[Petr Pisar]

On Fri, Dec 11, 2015 at 01:22:48PM +0200, Eli Zaretskii wrote:
> > Date: Thu, 10 Dec 2015 01:12:37 +0100
> > From: Ángel González 
> > Cc: bug-wget 
> > 
> > On 09/12/15 03:06, Random Coder wrote:
> > > I'm not sure if the wget maintainers would be interested, but I've
> > > been carrying this patch around in my private builds of wget for a
> > > while.  It allows wget to load SSL certs from the default Windows cert
> > > store.
> > >
> > > The patch itself is fairly straightforward, but as it changes the
> > > default SSL behavior, and no care was taken to follow coding convents
> > > when I wrote it, so it's probably not ready for inclusion in the
> > > codebase.  Still, if it's useful, feel free to use it for ideas.
> > Wow, supporting the OS store would certainly be very cool.
> > 
> > I would probably move it to windows.c and attempt to make it also work 
> > in gnutls, but in general it looks good.
> 
> Wget compiled with GnuTLS already supports this feature: it calls
> gnutls_certificate_set_x509_system_trust when the GnuTLS library
> supports that.  gnutls_certificate_set_x509_system_trust does
> internally what the proposed patch does.
> 
> So I think this code should indeed go only to openssl.c, as gnutls.c
> already has its equivalent.
> 
AFAIK OpenSSL source contains crypto engine that delegates all operations
to Windows native cryptographical subsystem. It's only matter of default
configuration.

-- Petr


signature.asc
Description: PGP signature

[Bug-wget] wget through proxy server to get dat from ftp.nai.com

[Hii Sing Chung]

Hi,

 

Appreciate your help.

I used this command:

Wget.exe �Ce http_proxy=sophosproxy:8080 E �Cproxy-user=myuser �C
proxy-password=mypassword ftp://ftp.nai.com/virusdefs/4.x/*dat.exe

 

I got this response:

 

Warning: wildcards not supported in HTTP.

Connecting to sophosproxy…..connected.

Proxy request sent, awaiting response… 500 File not found

Error 502: Host not found.

 

Most probable reason is because it doesn’t accept *.dat.exe in the command
line. As the dat files changes name everyday, how do I ensure I can download
whatever files that is under the 4.x directory?

 

Best regards,

Hii Sing Chung许新春

 

[Bug-wget] wget does not explicitly close the session when exiting

[Yoriyuki Yamagata]

Dear list,

I noticed that wget does not explicitly close the session when exiting.  
Looking dtruss output on Mac OS X, 

73072/0x11a24f:  read(0x6, "hts reserved.\n\t\t Privacy Policy Terms of Use \n\t\t Privacy Policy Terms of Use http://staff.aist.go.jp/yoriyuki.yamagata/en/

Re: [Bug-wget] Fixing Test-k for Cygwin (and hopefully Windows)

[Tim Ruehsen]

On Friday 11 December 2015 14:43:32 Gisle Vanem wrote:
> > Test-k fails on Cygwin with LC_ALL=C (which is the default for 'make
> > check').
> > 
> > With this patch the test works on CygWin and on Linux.
> > 
> > Here is a patch that *should* fix it also for 'plain' MS Windows.
> > Could anyone test this on Windows (without CygWin) ?
> 
> From perl.exe -d -d Test-k.px:
> 
> main::(Test-k.px:10):   my $osname = $^O;
> 
> SetConsoleMode failed, LastError=|6| at
> f:/util/StrawberryPerl/perl/vendor/lib/Term/ReadKey.pm line 309. at
> f:/util/StrawberryPerl/perl/vendor/lib/Term/ReadLine/readline.pm line 1581.
> readline::readline("  DB<1> ") called at
> f:/util/StrawberryPerl/perl/vendor/lib/Term/ReadLine/Perl.pm line 11
> Term::ReadLine::Perl::readline(Term::ReadLine::Perl=ARRAY(0xc2a6ec), " 
> DB<1> ") called at F:/util/StrawberryPer l/perl/lib/perl5db.pl line 7284
> DB::readline("  DB<1> ") called at
> F:/util/StrawberryPerl/perl/lib/perl5db.pl line 1831
> DB::_DB__read_next_cmd(undef) called at
> F:/util/StrawberryPerl/perl/lib/perl5db.pl line 2744 DB::DB called at
> Test-k.px line 10
>   DB<1> r
> OS=MSWin32
> Running test Test-k
> Cannot fork at Test-k.px line 90.
>  at WgetTests.pm line 410.
> WgetTest::_fork_and_launch_server(HTTPTest=HASH(0x33e634c)) called
> at WgetTests.pm line 111 WgetTest::run(HTTPTest=HASH(0x33e634c)) called at
> Test-k.px line 90
> 
> This has never work outside *nix or Cygwin I think.

Thanks for testing, Gisle !

That means the perl test suite is broken on Windows - thus no test suite at 
all on Windows !? Perl is not able to fork on Windows ?

Can anyone confirm that ?

Regards, Tim

Re: [Bug-wget] Fixing Test-k for Cygwin (and hopefully Windows)

[Gisle Vanem]

Tim Ruehsen wrote:

> That means the perl test suite is broken on Windows - thus no test suite at 
> all on Windows !? Perl is not able to fork on Windows ?

An option could be to use Perlfork:
  http://perldoc.perl.org/perlfork.html

But I'm no Perl expert.

-- 
--gv

Re: [Bug-wget] Marking Release v1.17.1?

[Giuseppe Scrivano]

Jernej Simončič  writes:

> On Tuesday, December 8, 2015, 21:45:31, Darshit Shah wrote:
>
>> With my last set of patches, we have fixed all the issues reported / 
>> identified
>> after the 1.17 release. Hence, maybe we should consider releasing a 1.17.1
>> bugfix release?
>
> Sorry about reporting this late, it forgot about it a bit: when I was
> preparing my Windows release, I noticed that metalink support first
> didn't want to compile (due to missing sys/errno.h - just #ifdefing it
> out didn't seem to break anything), and then that hash validation
> failed, because the file was being opened in text mode. I did a quick
> and dirty fix for my build
> ,
> but this probably isn't correct for non-Windows systems.

Thanks for the patch, one piece is already present upstream, I will
commit the missing part:

>From bf56bf4560cef3c1591487e7df1f2cb5e5ad0303 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jernej=20Simon=C4=8Di=C4=8D?=
 
Date: Fri, 11 Dec 2015 09:58:30 +0100
Subject: [PATCH] * src/metalink.c: Specify 'rb' as mode to open file

---
 src/metalink.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/metalink.c b/src/metalink.c
index 986303d..25737b3 100644
--- a/src/metalink.c
+++ b/src/metalink.c
@@ -168,7 +168,7 @@ retrieve_from_metalink (const metalink_t* metalink)
   FILE *local_file;
 
   /* Check the digest.  */
-  local_file = fopen (filename, "r");
+  local_file = fopen (filename, "rb");
   if (!local_file)
 {
   logprintf (LOG_NOTQUIET, _("Could not open downloaded 
file.\n"));
-- 
2.5.0



> Another thing that would be nice to include is the taskbar progressbar
> patch:
> 
> This one is based on patch by Ángel González and tbprogress.c from
> Gisle Vanem.

Darshit, is there anything left on the progress bar?

I will probably tag 1.17.1 later today.


Regards,
Giuseppe

Re: [Bug-wget] GNU wget 1.17.1 released

[Andries E. Brouwer]

On Fri, Dec 11, 2015 at 08:22:23PM +0100, Giuseppe Scrivano wrote:
> Hello,
> 
> I am pleased to announce the new version of GNU wget.  We consider it a
> bug fixes release as it addresses issues found in 1.17, which contained
> quite a few new features.
> 
> Please report any problem you may experience to the bug-wget@gnu.org
> mailing list.

Four months ago I mentioned two bugs:
1. Non-ASCII filenames are mistreated
2. The progress bar is broken when the filename is non-ASCII.
And I provided patches that fix 2, and fix 1 on Unix.

Tim Ruehsen polished the second patch a bit more, and then
nothing more was heard about it.
The first patch was for Unix, and there was some amount of discussion
of the Windows situation with Eli Zaretskii. But nobody offered
a Windows patch, and my patch is OK, but Unix-only.

Now that wget-1.17.1 is out, let me try it on the Russian Wikipedia page
for the page "heart", Сердце, the same example we used last August.

With my patch I get:

Saving to: ‘Сердце’

With wget-1.17.1 I get:

% wget/wget-1.17.1/src/wget 
https://ru.wikipedia.org/wiki/%D0%A1%D0%B5%D1%80%D0%B4%D1%86%D0%B5

Saving to: ‘Се\321%80д\321%86е’.
Here wget saves to a name that is not a legal name on this filesystem.
The progress bar is still broken and contains illegal characters.

So, I find that wget-1.17.1 is still broken, and no good for downloading
files with UTF-8 filename from an UTF-8 site to a local UTF-8 site.
There is some default built-in ISO 8859-* ugliness.

Andries

Re: [Bug-wget] Marking Release v1.17.1?

[Darshit Shah]

On 12/11, Giuseppe Scrivano wrote:

Jernej Simončič  writes:


On Tuesday, December 8, 2015, 21:45:31, Darshit Shah wrote:


With my last set of patches, we have fixed all the issues reported / identified
after the 1.17 release. Hence, maybe we should consider releasing a 1.17.1
bugfix release?


Sorry about reporting this late, it forgot about it a bit: when I was
preparing my Windows release, I noticed that metalink support first
didn't want to compile (due to missing sys/errno.h - just #ifdefing it
out didn't seem to break anything), and then that hash validation
failed, because the file was being opened in text mode. I did a quick
and dirty fix for my build
,
but this probably isn't correct for non-Windows systems.


Thanks for the patch, one piece is already present upstream, I will
commit the missing part:

From bf56bf4560cef3c1591487e7df1f2cb5e5ad0303 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jernej=20Simon=C4=8Di=C4=8D?=

Date: Fri, 11 Dec 2015 09:58:30 +0100
Subject: [PATCH] * src/metalink.c: Specify 'rb' as mode to open file

---
src/metalink.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/metalink.c b/src/metalink.c
index 986303d..25737b3 100644
--- a/src/metalink.c
+++ b/src/metalink.c
@@ -168,7 +168,7 @@ retrieve_from_metalink (const metalink_t* metalink)
  FILE *local_file;

  /* Check the digest.  */
-  local_file = fopen (filename, "r");
+  local_file = fopen (filename, "rb");
  if (!local_file)
{
  logprintf (LOG_NOTQUIET, _("Could not open downloaded 
file.\n"));
--
2.5.0




Another thing that would be nice to include is the taskbar progressbar
patch:

This one is based on patch by Ángel González and tbprogress.c from
Gisle Vanem.


Darshit, is there anything left on the progress bar?

This patch affects Windows only. I have no way to test it currently, and it 
uses Windows-specific constructs that I'm not aware of. I'll have to set up a 
Windows VM soon for testing these patches.



I will probably tag 1.17.1 later today.


Regards,
Giuseppe


--
Thanking You,
Darshit Shah


signature.asc
Description: PGP signature

Re: [Bug-wget] wget through proxy server to get dat from ftp.nai.com

[Tim Ruehsen]

Give -r a try, like in

wget -r ...proxy settings... ftp://ftp.nai.com/virusdefs/4.x/

Adding -N should avoid re-downloading of already existing files.

Tim

On Friday 11 December 2015 13:16:29 Hii Sing Chung wrote:
> Hi,
> 
> 
> 
> Appreciate your help.
> 
> I used this command:
> 
> Wget.exe –e http_proxy=sophosproxy:8080 E –proxy-user=myuser –
> proxy-password=mypassword ftp://ftp.nai.com/virusdefs/4.x/*dat.exe
> 
> 
> 
> I got this response:
> 
> 
> 
> Warning: wildcards not supported in HTTP.
> 
> Connecting to sophosproxy…..connected.
> 
> Proxy request sent, awaiting response… 500 File not found
> 
> Error 502: Host not found.
> 
> 
> 
> Most probable reason is because it doesn’t accept *.dat.exe in the command
> line. As the dat files changes name everyday, how do I ensure I can download
> whatever files that is under the 4.x directory?
> 
> 
> 
> Best regards,
> 
> Hii Sing Chung许新春

[Bug-wget] buildbot failure in OpenCSW Buildbot on wget-solaris10-i386

[buildbot]

The Buildbot has detected a new failure on builder wget-solaris10-i386 while 
building wget.
Full details are available at:
 https://buildfarm.opencsw.org/buildbot/builders/wget-solaris10-i386/builds/111

Buildbot URL: https://buildfarm.opencsw.org/buildbot/

Buildslave for this Build: unstable10x

Build Reason: scheduler
Build Source Stamp: [branch master] f65f522f58606f125535306e712c2c29921ee8b4
Blamelist: Giuseppe Scrivano 

BUILD FAILED: failed shell_2 shell_3

sincerely,
 -The Buildbot

[Bug-wget] GNU wget 1.17.1 released

[Giuseppe Scrivano]

Hello,

I am pleased to announce the new version of GNU wget.  We consider it a
bug fixes release as it addresses issues found in 1.17, which contained
quite a few new features.

It is available for download here:

ftp://ftp.gnu.org/gnu/wget/wget-1.17.1.tar.gz
ftp://ftp.gnu.org/gnu/wget/wget-1.17.1.tar.xz

and the GPG detached signatures using the key E163E1EA:

ftp://ftp.gnu.org/gnu/wget/wget-1.17.1.tar.gz.sig
ftp://ftp.gnu.org/gnu/wget/wget-1.17.1.tar.xz.sig

To reduce load on the main server, you can use this redirector service
which automatically redirects you to a mirror:

http://ftpmirror.gnu.org/wget/wget-1.17.1.tar.gz
http://ftpmirror.gnu.org/wget/wget-1.17.1.tar.xz

Noteworthy changes:

* Fix compile error when IPv6 is disabled or SSL is not present.

* Fix HSTS memory leak.

* Fix progress output in non-C locales.

* Fix SIGSEGV when -N and --content-disposition are used together.

* Add --check-certificate=quiet to tell wget to not print any warning about
  invalid certificates.

Please report any problem you may experience to the bug-wget@gnu.org
mailing list.

Have fun!
Giuseppe

[Bug-wget] buildbot failure in OpenCSW Buildbot on wget-solaris10-sparc

[buildbot]

The Buildbot has detected a new failure on builder wget-solaris10-sparc while 
building wget.
Full details are available at:
 https://buildfarm.opencsw.org/buildbot/builders/wget-solaris10-sparc/builds/116

Buildbot URL: https://buildfarm.opencsw.org/buildbot/

Buildslave for this Build: unstable10s

Build Reason: scheduler
Build Source Stamp: [branch master] f65f522f58606f125535306e712c2c29921ee8b4
Blamelist: Giuseppe Scrivano 

BUILD FAILED: failed shell_2 shell_3

sincerely,
 -The Buildbot