Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-29 Thread Heikki Linnakangas 

On 07/29/2015 10:13 AM, Michael Paquier wrote:

On Sat, Jul 25, 2015 at 10:54 PM, Michael Paquier
michael.paqu...@gmail.com wrote:

An updated patch is attached.


Attached is v9, that fixes conflicts with 01f6bb4 and recent commits
that added TAP tests in pg_basebackup series.


Thanks, committed with some more tweaking. Peter just added a 
slurp_file() function to TestLib.pm, so I used that to replace the call 
to 'cat' instead of my previous snippet. I also fixed the issue in the 
pg_basebackup test, that LSN is not necessarily 8 characters long, 
slightly differently. Calling pg_xlogfile_name seemed a bit indirect.


I expanded the documentation on how to download and install IPC::Run. I 
instructed to add PERL5LIB to buildenv.pl, pointing it to the extracted 
IPC-Run-0.94/lib directory. Your phrasing of putting it into PERL5LIB 
was pretty vague, and in fact the PERL5LIB environment variable was 
overwritten in vcregress.pl, so just setting the PERL5LIB variable 
wouldn't have worked. I changed vcregress.pl to not do that.


I considered moving the instructions on downloading and installing 
IPC::Run in the Requirements section, instead of the Running the 
Regression Tests section. But there are additional instructions on 
downloading and installing more dependencies in the Building the 
Documentation section too. The other dependencies in the Requirements 
section affect the built binaries. Well, except for the Diff utility. 
We're not totally consistent, but I think it's OK as it is.



Also I added an equivalent of --enable-tap-tests for this MSVC stuff,
removed afterwards by Heikki. Do we want it or not?


On Unix, that option controls whether make check-world runs the TAP 
tests, but there is no equivalent of check-world on Windows. Maybe 
there should be, but as long as there isn't, the only thing that the 
option did was to stop vcregress tapcheck from working, if didn't 
enable it in the config file first. That seems silly, so I removed it. 
We'll need it if we add an equivalent of make check-world to 
vcregress, but let's cross that bridge when we get there.


- Heikki



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-29 Thread Michael Paquier 
On Thu, Jul 30, 2015 at 1:37 AM, Heikki Linnakangas hlinn...@iki.fi wrote:
 On 07/29/2015 10:13 AM, Michael Paquier wrote:

 On Sat, Jul 25, 2015 at 10:54 PM, Michael Paquier
 michael.paqu...@gmail.com wrote:

 An updated patch is attached.


 Attached is v9, that fixes conflicts with 01f6bb4 and recent commits
 that added TAP tests in pg_basebackup series.


 Thanks, committed with some more tweaking. Peter just added a slurp_file()
 function to TestLib.pm, so I used that to replace the call to 'cat' instead
 of my previous snippet. I also fixed the issue in the pg_basebackup test,
 that LSN is not necessarily 8 characters long, slightly differently. Calling
 pg_xlogfile_name seemed a bit indirect.

Thanks!
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-29 Thread Michael Paquier 
On Sat, Jul 25, 2015 at 10:54 PM, Michael Paquier
michael.paqu...@gmail.com wrote:
 An updated patch is attached.

Attached is v9, that fixes conflicts with 01f6bb4 and recent commits
that added TAP tests in pg_basebackup series.
-- 
Michael


0001-TAP-tests-for-MSVC.patch
Description: binary/octet-stream

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-25 Thread Michael Paquier 
On Sat, Jul 25, 2015 at 4:14 PM, Noah Misch n...@leadboat.com wrote:
 On Fri, Jul 24, 2015 at 08:27:42PM +0300, Heikki Linnakangas wrote:
 On 06/25/2015 07:40 AM, Michael Paquier wrote:
 Attached is v7, rebased on 0b157a0.

 Thanks! I fiddled with this a bit more, to centralize more of the
 platform-dependent stuff to RewindTest.pm. Also, Windows doesn't have cat
 and touch if you haven't installed MinGW, so I replaced those calls with
 built-in perl code.

 My main priority for this patch is to not reintroduce CVE-2014-0067.  The
 patch is operating in a security minefield:

Thanks!

 --- a/src/bin/pg_ctl/t/001_start_stop.pl
 +++ b/src/bin/pg_ctl/t/001_start_stop.pl
 @@ -15,13 +15,9 @@ command_exit_is([ 'pg_ctl', 'start', '-D', 
 $tempdir/nonexistent ],

  command_ok([ 'pg_ctl', 'initdb', '-D', $tempdir/data ], 'pg_ctl initdb');
  command_ok(
 - [   $ENV{top_builddir}/src/test/regress/pg_regress, '--config-auth',
 + [ $ENV{PG_REGRESS}, '--config-auth',
   $tempdir/data ],
   'configure authentication');
 -open CONF, $tempdir/data/postgresql.conf;
 -print CONF listen_addresses = ''\n;
 -print CONF unix_socket_directories = '$tempdir_short'\n;
 -close CONF;
  command_ok([ 'pg_ctl', 'start', '-D', $tempdir/data, '-w' ],
   'pg_ctl start -w');

 Since this series of tests doesn't use standard_initdb, the deleted lines
 remain necessary.

Added with a switch on $config{osname}.

 @@ -303,7 +297,6 @@ sub run_pg_rewind
   }
   else
   {
 -
   # Cannot come here normally

 Unrelated change.

Removed.

  sub standard_initdb
  {
   my $pgdata = shift;
   system_or_bail('initdb', '-D', $pgdata, '-A' , 'trust', '-N');
 - system_or_bail($ENV{top_builddir}/src/test/regress/pg_regress,
 - '--config-auth', $pgdata);
 + system_or_bail($ENV{PG_REGRESS}, '--config-auth', $pgdata);
 +
 + my $tempdir_short = tempdir_short;
 +
 + open CONF, $pgdata/postgresql.conf;
 + print CONF \n# Added by TestLib.pm)\n;
 + if ($Config{osname} eq MSWin32)
 + {
 + print CONF listen_addresses = '127.0.0.1'\n;
 + }
 + else
 + {
 + print CONF unix_socket_directories = '$tempdir_short'\n;

 This second branch needs listen_addresses=''; it doesn't help to secure the
 socket directory if the server still listens on localhost.

Yep. Agreed.

 +sub configure_hba_for_replication
 +{
 + my $pgdata = shift;
 +
 + open HBA, $pgdata/pg_hba.conf;
 + print HBA \n# Allow replication (set up by TestLib.pm)\n;
 + if ($Config{osname} ne MSWin32)
 + {
 + print HBA local replication all trust\n;
 + }
 + else
 + {
 + print HBA host replication all 127.0.0.1/32 sspi 
 include_realm=1 map=regress\n;
 + print HBA host replication all ::1/128 sspi include_realm=1 
 map=regress\n;

 The second line will make the server fail to start on non-IPv6 systems.  You
 shouldn't need it if the clients always connect to 127.0.0.1, not localhost.

Done.

 + configure_for_replication($tempdir/pgdata);

 That function name appears nowhere else.

This looks like dead code to me. Hence removed.

 +sub tapcheck
 +{
 + InstallTemp();
 +
 + my @args = ( prove, --verbose, t/*.pl);
 +
 + $ENV{PATH} = $tmp_installdir/bin;$ENV{PATH};
 + $ENV{PERL5LIB} = $topdir/src/test/perl;$ENV{PERL5LIB};
 + $ENV{PG_REGRESS} = $topdir/$Config/pg_regress/pg_regress;
 +
 + # Find out all the existing TAP tests by simply looking for t/
 + # in the tree.

 This target shall not run the SSL suite, for the same reason check-world shall
 not run it.  We could offer a distinct vcregress.pl target for TAP suites
 excluded from check-world.

OK, for the sake of duplicating what GNU does, let's simply ignore it then.

Also I added an equivalent of --enable-tap-tests for this MSVC stuff,
removed afterwards by Heikki. Do we want it or not?

An updated patch is attached.
-- 
Michael
From 51d6d553742753a3389fcb67778662cb24274508 Mon Sep 17 00:00:00 2001
From: Michael Paquier mich...@otacoo.com
Date: Sat, 25 Jul 2015 22:50:49 +0900
Subject: [PATCH] Make TAP tests work on Windows.

On Windows, use listen_address=127.0.0.1 to allow connections. We were
already using pg_regress --config-auth to set up HBA appropriately.
The standard_initdb helper function now sets up the server's
unix_socket_directories or listen_addresses in the config file, so that
they don't need to be specified in the pg_ctl command line anymore. That
way, the pg_ctl invocations in test programs don't need to differ between
Windows and Unix.

Add another helper function to configure the server's pg_hba.conf to
allow replication connections. The configuration is done similarly to
pg_regress --config-auth: trust on domain sockets on Unix, and SSPI
authentication on Windows.

Replace calls to cat and touch programs with built-in perl code.

Add vcregress tapcheck command for launching the tests on Windows.

Michael

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-25 Thread Noah Misch 
On Fri, Jul 24, 2015 at 08:27:42PM +0300, Heikki Linnakangas wrote:
 On 06/25/2015 07:40 AM, Michael Paquier wrote:
 Attached is v7, rebased on 0b157a0.
 
 Thanks! I fiddled with this a bit more, to centralize more of the
 platform-dependent stuff to RewindTest.pm. Also, Windows doesn't have cat
 and touch if you haven't installed MinGW, so I replaced those calls with
 built-in perl code.

My main priority for this patch is to not reintroduce CVE-2014-0067.  The
patch is operating in a security minefield:

 --- a/src/bin/pg_ctl/t/001_start_stop.pl
 +++ b/src/bin/pg_ctl/t/001_start_stop.pl
 @@ -15,13 +15,9 @@ command_exit_is([ 'pg_ctl', 'start', '-D', 
 $tempdir/nonexistent ],
  
  command_ok([ 'pg_ctl', 'initdb', '-D', $tempdir/data ], 'pg_ctl initdb');
  command_ok(
 - [   $ENV{top_builddir}/src/test/regress/pg_regress, '--config-auth',
 + [ $ENV{PG_REGRESS}, '--config-auth',
   $tempdir/data ],
   'configure authentication');
 -open CONF, $tempdir/data/postgresql.conf;
 -print CONF listen_addresses = ''\n;
 -print CONF unix_socket_directories = '$tempdir_short'\n;
 -close CONF;
  command_ok([ 'pg_ctl', 'start', '-D', $tempdir/data, '-w' ],
   'pg_ctl start -w');

Since this series of tests doesn't use standard_initdb, the deleted lines
remain necessary.

 @@ -303,7 +297,6 @@ sub run_pg_rewind
   }
   else
   {
 -
   # Cannot come here normally

Unrelated change.

  sub standard_initdb
  {
   my $pgdata = shift;
   system_or_bail('initdb', '-D', $pgdata, '-A' , 'trust', '-N');
 - system_or_bail($ENV{top_builddir}/src/test/regress/pg_regress,
 - '--config-auth', $pgdata);
 + system_or_bail($ENV{PG_REGRESS}, '--config-auth', $pgdata);
 +
 + my $tempdir_short = tempdir_short;
 +
 + open CONF, $pgdata/postgresql.conf;
 + print CONF \n# Added by TestLib.pm)\n;
 + if ($Config{osname} eq MSWin32)
 + {
 + print CONF listen_addresses = '127.0.0.1'\n;
 + }
 + else
 + {
 + print CONF unix_socket_directories = '$tempdir_short'\n;

This second branch needs listen_addresses=''; it doesn't help to secure the
socket directory if the server still listens on localhost.

 +sub configure_hba_for_replication
 +{
 + my $pgdata = shift;
 +
 + open HBA, $pgdata/pg_hba.conf;
 + print HBA \n# Allow replication (set up by TestLib.pm)\n;
 + if ($Config{osname} ne MSWin32)
 + {
 + print HBA local replication all trust\n;
 + }
 + else
 + {
 + print HBA host replication all 127.0.0.1/32 sspi 
 include_realm=1 map=regress\n;
 + print HBA host replication all ::1/128 sspi include_realm=1 
 map=regress\n;

The second line will make the server fail to start on non-IPv6 systems.  You
shouldn't need it if the clients always connect to 127.0.0.1, not localhost.

 + configure_for_replication($tempdir/pgdata);

That function name appears nowhere else.

 +sub tapcheck
 +{
 + InstallTemp();
 +
 + my @args = ( prove, --verbose, t/*.pl);
 +
 + $ENV{PATH} = $tmp_installdir/bin;$ENV{PATH};
 + $ENV{PERL5LIB} = $topdir/src/test/perl;$ENV{PERL5LIB};
 + $ENV{PG_REGRESS} = $topdir/$Config/pg_regress/pg_regress;
 +
 + # Find out all the existing TAP tests by simply looking for t/
 + # in the tree.

This target shall not run the SSL suite, for the same reason check-world shall
not run it.  We could offer a distinct vcregress.pl target for TAP suites
excluded from check-world.

nm


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-25 Thread Michael Paquier 
On Sat, Jul 25, 2015 at 2:27 AM, Heikki Linnakangas hlinn...@iki.fi wrote:
 On 06/25/2015 07:40 AM, Michael Paquier wrote:

 On Tue, May 26, 2015 at 3:39 PM, Michael Paquier wrote:

 Here is v6, a rebased version on HEAD (79f2b5d). There were some
 conflicts with the indentation and some other patches related to
 pg_rewind and initdb's tests.


 Attached is v7, rebased on 0b157a0.


 Thanks! I fiddled with this a bit more, to centralize more of the
 platform-dependent stuff to RewindTest.pm. Also, Windows doesn't have cat
 and touch if you haven't installed MinGW, so I replaced those calls with
 built-in perl code.

 Can you double-check that the attached still works in your environment? It
 works for me now.

Sure, I'll double check and update your last version as necessary.

 We need to put some instructions in the docs on how to install IPC::Run on
 Windows. I can write up something unless you're eager to.

I'll do it. That's fine, but it is going to be closer to a ninja
version of what you did.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-24 Thread Heikki Linnakangas 

On 06/25/2015 07:40 AM, Michael Paquier wrote:

On Tue, May 26, 2015 at 3:39 PM, Michael Paquier wrote:

Here is v6, a rebased version on HEAD (79f2b5d). There were some
conflicts with the indentation and some other patches related to
pg_rewind and initdb's tests.


Attached is v7, rebased on 0b157a0.


Thanks! I fiddled with this a bit more, to centralize more of the 
platform-dependent stuff to RewindTest.pm. Also, Windows doesn't have 
cat and touch if you haven't installed MinGW, so I replaced those 
calls with built-in perl code.


Can you double-check that the attached still works in your environment? 
It works for me now.


Note: I had some trouble installing IPC::Run on my system, with 
ActiveState Perl and MSVC. There is no PPM package of that for Windows, 
so I had to do cpan install IPC::Run. That downloaded the MinGW C 
compiler and make utility, which took a while. But some of the IPC::Run 
regression tests failed, and the installation was aborted. I forced my 
way through that notest install IPC::Run. The next obstacle was that 
vcregress anything no longer worked. It complained about finding 
some function in the Install module. Turns out that when it installed 
the C compiler and make utility, it also installed a module called 
install from cpan, which has the same name as the PostgreSQL 
Install.pm module. We really should rename our module. I got through 
that by manually removing the system install.pm module from the perl 
installation's site directory. But after that, it worked great :-).


We need to put some instructions in the docs on how to install IPC::Run 
on Windows. I can write up something unless you're eager to.


- Heikki

From 1793f273cb2d849c9fc63483ab485fba29c114d6 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas heikki.linnakangas@iki.fi
Date: Fri, 24 Jul 2015 17:39:24 +0300
Subject: [PATCH] Make TAP tests work on Windows.

On Windows, use listen_address=127.0.0.1 to allow connections. We were
already using pg_regress --config-auth to set up HBA appropriately.
The standard_initdb helper function now sets up the server's
unix_socket_directories or listen_addresses in the config file, so that
they don't need to be specified in the pg_ctl command line anymore. That
way, the pg_ctl invocations in test programs don't need to differ between
Windows and Unix.

Add another helper function to configure the server's pg_hba.conf to
allow replication connections. The configuration is done similarly to
pg_regress --config-auth: trust on domain sockets on Unix, and SSPI
authentication on Windows.

Replace calls to cat and touch programs with built-in perl code.

Add vcregress tapcheck command for launching the tests on Windows.

Michael Paquier

diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windows.sgml
index d154b44..2047790 100644
--- a/doc/src/sgml/install-windows.sgml
+++ b/doc/src/sgml/install-windows.sgml
@@ -439,6 +439,7 @@ $ENV{CONFIG}=Debug;
 userinputvcregress modulescheck/userinput
 userinputvcregress ecpgcheck/userinput
 userinputvcregress isolationcheck/userinput
+userinputvcregress tapcheck/userinput
 userinputvcregress upgradecheck/userinput
 /screen
 
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index e2f7211..59c9eed 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -339,7 +339,7 @@ endef
 
 define prove_check
 rm -rf $(srcdir)/tmp_check/log
-cd $(srcdir)  TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir)  TESTDIR='$(CURDIR)' PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
 endef
 
 else
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index e47c3a0..40b6b76 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -1,6 +1,7 @@
 use strict;
 use warnings;
 use Cwd;
+use Config;
 use TestLib;
 use Test::More tests = 35;
 
@@ -25,11 +26,7 @@ if (open BADCHARS, $tempdir/pgdata/FOO\xe0\xe0\xe0BAR)
 	close BADCHARS;
 }
 
-open HBA, $tempdir/pgdata/pg_hba.conf;
-print HBA local replication all trust\n;
-print HBA host replication all 127.0.0.1/32 trust\n;
-print HBA host replication all ::1/128 trust\n;
-close HBA;
+configure_hba_for_replication $tempdir/pgdata;
 system_or_bail 'pg_ctl', '-D', $tempdir/pgdata, 'reload';
 
 command_fails(
@@ -57,61 +54,7 @@ command_ok([ 'pg_basebackup', '-D', $tempdir/tarbackup, '-Ft' ],
 	'tar format');
 ok(-f $tempdir/tarbackup/base.tar, 'backup tar was created');
 
-my $superlongname = superlongname_ . (x x 100);
-
-system_or_bail 'touch', $tempdir/pgdata/$superlongname;
-command_fails([ 'pg_basebackup', '-D', $tempdir/tarbackup_l1, '-Ft' ],
-	'pg_basebackup tar with long name fails');
-unlink $tempdir/pgdata/$superlongname;
-
-# Create a temporary

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-24 Thread Andrew Dunstan 


On 07/24/2015 01:27 PM, Heikki Linnakangas wrote:

On 06/25/2015 07:40 AM, Michael Paquier wrote:

On Tue, May 26, 2015 at 3:39 PM, Michael Paquier wrote:

Here is v6, a rebased version on HEAD (79f2b5d). There were some
conflicts with the indentation and some other patches related to
pg_rewind and initdb's tests.


Attached is v7, rebased on 0b157a0.


Thanks! I fiddled with this a bit more, to centralize more of the 
platform-dependent stuff to RewindTest.pm. Also, Windows doesn't have 
cat and touch if you haven't installed MinGW, so I replaced those 
calls with built-in perl code.


Can you double-check that the attached still works in your 
environment? It works for me now.


Note: I had some trouble installing IPC::Run on my system, with 
ActiveState Perl and MSVC. There is no PPM package of that for 
Windows, so I had to do cpan install IPC::Run. That downloaded the 
MinGW C compiler and make utility, which took a while. But some of the 
IPC::Run regression tests failed, and the installation was aborted. I 
forced my way through that notest install IPC::Run. The next 
obstacle was that vcregress anything no longer worked. It 
complained about finding some function in the Install module. Turns 
out that when it installed the C compiler and make utility, it also 
installed a module called install from cpan, which has the same name 
as the PostgreSQL Install.pm module. We really should rename our 
module. I got through that by manually removing the system install.pm 
module from the perl installation's site directory. But after that, it 
worked great :-).


We need to put some instructions in the docs on how to install 
IPC::Run on Windows. I can write up something unless you're eager to.



AFAIK all you need to do is put Run.pm in the right place. It doesn't 
need any building, IIRC, and the current version can be got from 
http://cpansearch.perl.org/src/TODDR/IPC-Run-0.94/lib/IPC/Run.pm



cheers

andrew


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-24 Thread Michael Paquier 
On Sat, Jul 25, 2015 at 3:27 AM, Andrew Dunstan and...@dunslane.net wrote:

 On 07/24/2015 01:27 PM, Heikki Linnakangas wrote:

 On 06/25/2015 07:40 AM, Michael Paquier wrote:

 On Tue, May 26, 2015 at 3:39 PM, Michael Paquier wrote:

 Here is v6, a rebased version on HEAD (79f2b5d). There were some
 conflicts with the indentation and some other patches related to
 pg_rewind and initdb's tests.


 Attached is v7, rebased on 0b157a0.


 Thanks! I fiddled with this a bit more, to centralize more of the
 platform-dependent stuff to RewindTest.pm. Also, Windows doesn't have cat
 and touch if you haven't installed MinGW, so I replaced those calls with
 built-in perl code.

 Can you double-check that the attached still works in your environment? It
 works for me now.

 Note: I had some trouble installing IPC::Run on my system, with
 ActiveState Perl and MSVC. There is no PPM package of that for Windows, so I
 had to do cpan install IPC::Run. That downloaded the MinGW C compiler and
 make utility, which took a while. But some of the IPC::Run regression tests
 failed, and the installation was aborted. I forced my way through that
 notest install IPC::Run. The next obstacle was that vcregress anything
 no longer worked. It complained about finding some function in the Install
 module. Turns out that when it installed the C compiler and make utility, it
 also installed a module called install from cpan, which has the same name
 as the PostgreSQL Install.pm module. We really should rename our module. I
 got through that by manually removing the system install.pm module from the
 perl installation's site directory. But after that, it worked great :-).

 We need to put some instructions in the docs on how to install IPC::Run on
 Windows. I can write up something unless you're eager to.



 AFAIK all you need to do is put Run.pm in the right place. It doesn't need
 any building, IIRC, and the current version can be got from
 http://cpansearch.perl.org/src/TODDR/IPC-Run-0.94/lib/IPC/Run.pm

Yeah, that's exactly what I did myself and put it in a path of
PERL5LIB, and it just worked.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-07-09 Thread Michael Paquier 
On Thu, Jun 25, 2015 at 1:40 PM, Michael Paquier
michael.paqu...@gmail.com wrote:
 On Tue, May 26, 2015 at 3:39 PM, Michael Paquier wrote:
 Here is v6, a rebased version on HEAD (79f2b5d). There were some
 conflicts with the indentation and some other patches related to
 pg_rewind and initdb's tests.

 Attached is v7, rebased on 0b157a0.

Attached is v8, rebased on 1ea0620, meaning that it includes all the
fancy improvements in log capture for TAP tests.
-- 
Michael
From e8db0a990ec02ddb5bdf2fd95d42b297360ecd55 Mon Sep 17 00:00:00 2001
From: Michael Paquier mich...@otacoo.com
Date: Thu, 9 Jul 2015 06:46:13 -0700
Subject: [PATCH] Add support for TAP tests on Windows

Nodes initialized by the TAP tests use SSPI to securely perform the
tests, and test scripts are patched in a couple of places to support
Windows grammar. In the case of MSVC, tests can be run with this
command:
vcregress tapcheck
---
 doc/src/sgml/install-windows.sgml|  1 +
 src/Makefile.global.in   |  2 +-
 src/bin/pg_basebackup/t/010_pg_basebackup.pl | 67 +---
 src/bin/pg_ctl/t/001_start_stop.pl   | 14 --
 src/bin/pg_ctl/t/002_status.pl   | 12 -
 src/bin/pg_rewind/RewindTest.pm  | 56 ---
 src/bin/scripts/t/020_createdb.pl|  3 ++
 src/test/perl/TestLib.pm | 19 +---
 src/tools/msvc/Solution.pm   |  1 +
 src/tools/msvc/clean.bat |  8 
 src/tools/msvc/config_default.pl |  1 +
 src/tools/msvc/vcregress.pl  | 48 +++-
 12 files changed, 176 insertions(+), 56 deletions(-)

diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windows.sgml
index d154b44..2047790 100644
--- a/doc/src/sgml/install-windows.sgml
+++ b/doc/src/sgml/install-windows.sgml
@@ -439,6 +439,7 @@ $ENV{CONFIG}=Debug;
 userinputvcregress modulescheck/userinput
 userinputvcregress ecpgcheck/userinput
 userinputvcregress isolationcheck/userinput
+userinputvcregress tapcheck/userinput
 userinputvcregress upgradecheck/userinput
 /screen
 
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8d1250d..4b9c529 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -338,7 +338,7 @@ endef
 
 define prove_check
 rm -rf $(srcdir)/tmp_check/log
-cd $(srcdir)  TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir)  TESTDIR='$(CURDIR)' TESTREGRESS='$(top_builddir)/src/test/regress/pg_regress' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
 endef
 
 else
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index e47c3a0..59e8cb4 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -1,8 +1,9 @@
 use strict;
 use warnings;
 use Cwd;
+use Config;
 use TestLib;
-use Test::More tests = 35;
+use Test::More tests = ($Config{osname} eq MSWin32) ? 25 : 35;
 
 program_help_ok('pg_basebackup');
 program_version_ok('pg_basebackup');
@@ -25,10 +26,20 @@ if (open BADCHARS, $tempdir/pgdata/FOO\xe0\xe0\xe0BAR)
 	close BADCHARS;
 }
 
+# Use SSPI on Windows, node has been initialized already accordingly
+# by pg_regress --config-auth.
 open HBA, $tempdir/pgdata/pg_hba.conf;
-print HBA local replication all trust\n;
-print HBA host replication all 127.0.0.1/32 trust\n;
-print HBA host replication all ::1/128 trust\n;
+if ($Config{osname} ne MSWin32)
+{
+	print HBA local replication all trust\n;
+	print HBA host replication all 127.0.0.1/32 trust\n;
+	print HBA host replication all ::1/128 trust\n;
+}
+else
+{
+	print HBA host replication all 127.0.0.1/32 sspi include_realm=1 map=regress\n;
+	print HBA host replication all ::1/128 sspi include_realm=1 map=regress\n;
+}
 close HBA;
 system_or_bail 'pg_ctl', '-D', $tempdir/pgdata, 'reload';
 
@@ -64,6 +75,33 @@ command_fails([ 'pg_basebackup', '-D', $tempdir/tarbackup_l1, '-Ft' ],
 	'pg_basebackup tar with long name fails');
 unlink $tempdir/pgdata/$superlongname;
 
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -T=/foo ],
+	'-T with empty old directory fails');
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -T/foo= ],
+	'-T with empty new directory fails');
+command_fails(
+	[   'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp',
+		-T/foo=/bar=/baz ],
+	'-T with multiple = fails');
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -Tfoo=/bar ],
+	'-T with old directory not absolute fails');
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -T/foo=bar ],
+	'-T with new directory not absolute fails');
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -Tfoo ],
+	'-T with invalid format fails');
+
+# Windows does not

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-06-24 Thread Michael Paquier 
On Tue, May 26, 2015 at 3:39 PM, Michael Paquier wrote:
 Here is v6, a rebased version on HEAD (79f2b5d). There were some
 conflicts with the indentation and some other patches related to
 pg_rewind and initdb's tests.

Attached is v7, rebased on 0b157a0.
-- 
Michael
From 4b960b29446c01ca2f1a44f0e545835d99ab87e6 Mon Sep 17 00:00:00 2001
From: Michael Paquier mich...@otacoo.com
Date: Mon, 20 Apr 2015 04:57:37 -0700
Subject: [PATCH 1/2] Add support for TAP tests on Windows

Nodes initialized by the TAP tests use SSPI to securely perform the
tests, and test scripts are patched in a couple of places to support
Windows grammar. In the case of MSVC, tests can be run with this
command:
vcregress tapcheck
---
 doc/src/sgml/install-windows.sgml  |  1 +
 src/Makefile.global.in |  2 +-
 src/bin/initdb/t/001_initdb.pl |  1 +
 src/bin/pg_basebackup/t/010_pg_basebackup.pl   | 67 --
 src/bin/pg_controldata/t/001_pg_controldata.pl |  5 +-
 src/bin/pg_ctl/t/001_start_stop.pl | 14 --
 src/bin/pg_ctl/t/002_status.pl | 12 -
 src/bin/pg_rewind/RewindTest.pm| 66 +++--
 src/bin/scripts/t/020_createdb.pl  |  3 ++
 src/test/perl/TestLib.pm   | 18 ---
 src/tools/msvc/Solution.pm |  1 +
 src/tools/msvc/config_default.pl   |  1 +
 src/tools/msvc/vcregress.pl| 48 +-
 13 files changed, 175 insertions(+), 64 deletions(-)

diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windows.sgml
index d154b44..2047790 100644
--- a/doc/src/sgml/install-windows.sgml
+++ b/doc/src/sgml/install-windows.sgml
@@ -439,6 +439,7 @@ $ENV{CONFIG}=Debug;
 userinputvcregress modulescheck/userinput
 userinputvcregress ecpgcheck/userinput
 userinputvcregress isolationcheck/userinput
+userinputvcregress tapcheck/userinput
 userinputvcregress upgradecheck/userinput
 /screen
 
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index c583b44..563d1d1 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -336,7 +336,7 @@ cd $(srcdir)  TESTDIR='$(CURDIR)' PATH=$(bindir):$$PATH PGPORT='6$(DEF_PGPOR
 endef
 
 define prove_check
-cd $(srcdir)  TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir)  TESTDIR='$(CURDIR)' TESTREGRESS='$(top_builddir)/src/test/regress/pg_regress' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
 endef
 
 else
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index 299dcf5..095cbf3 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -4,6 +4,7 @@
 
 use strict;
 use warnings;
+use Config;
 use TestLib;
 use Test::More tests = 14;
 
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index c8c9250..3fd297b 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -1,8 +1,9 @@
 use strict;
 use warnings;
 use Cwd;
+use Config;
 use TestLib;
-use Test::More tests = 35;
+use Test::More tests = ($Config{osname} eq MSWin32) ? 25 : 35;
 
 program_help_ok('pg_basebackup');
 program_version_ok('pg_basebackup');
@@ -25,10 +26,20 @@ if (open BADCHARS, $tempdir/pgdata/FOO\xe0\xe0\xe0BAR)
 	close BADCHARS;
 }
 
+# Use SSPI on Windows, node has been initialized already accordingly
+# by pg_regress --config-auth.
 open HBA, $tempdir/pgdata/pg_hba.conf;
-print HBA local replication all trust\n;
-print HBA host replication all 127.0.0.1/32 trust\n;
-print HBA host replication all ::1/128 trust\n;
+if ($Config{osname} ne MSWin32)
+{
+	print HBA local replication all trust\n;
+	print HBA host replication all 127.0.0.1/32 trust\n;
+	print HBA host replication all ::1/128 trust\n;
+}
+else
+{
+	print HBA host replication all 127.0.0.1/32 sspi include_realm=1 map=regress\n;
+	print HBA host replication all ::1/128 sspi include_realm=1 map=regress\n;
+}
 close HBA;
 system_or_bail 'pg_ctl', '-s', '-D', $tempdir/pgdata, 'reload';
 
@@ -64,6 +75,33 @@ command_fails([ 'pg_basebackup', '-D', $tempdir/tarbackup_l1, '-Ft' ],
 	'pg_basebackup tar with long name fails');
 unlink $tempdir/pgdata/$superlongname;
 
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -T=/foo ],
+	'-T with empty old directory fails');
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -T/foo= ],
+	'-T with empty new directory fails');
+command_fails(
+	[   'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp',
+		-T/foo=/bar=/baz ],
+	'-T with multiple = fails');
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -Tfoo=/bar ],
+	'-T with old directory not absolute fails');
+command_fails(
+	[ 'pg_basebackup', '-D',

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-05-26 Thread Michael Paquier 
On Fri, Apr 24, 2015 at 11:26 AM, Michael Paquier
michael.paqu...@gmail.com wrote:
 On Mon, Apr 20, 2015 at 9:01 PM, Michael Paquier
 michael.paqu...@gmail.com wrote:
 On Sun, Apr 19, 2015 at 10:01 PM, Michael Paquier
 michael.paqu...@gmail.com wrote:
 Note as well that this patch uses the following patches fixing
 independent issues:
 ...

 Attached is v4. I added a switch in config.pl to be consistent with
 ./configure and --enable-tap-tests.

 Attached is v5, rebased on HEAD (2c47fe16) after conflicts with
 dcae5fac and 54a16df0.

Here is v6, a rebased version on HEAD (79f2b5d). There were some
conflicts with the indentation and some other patches related to
pg_rewind and initdb's tests.
-- 
Michael
From 5879da14bbf20b5ac66b78f8beab09114bb2fa96 Mon Sep 17 00:00:00 2001
From: Michael Paquier mich...@otacoo.com
Date: Mon, 20 Apr 2015 04:57:37 -0700
Subject: [PATCH 1/2] Add support for TAP tests on Windows

Nodes initialized by the TAP tests use SSPI to securely perform the
tests, and test scripts are patched in a couple of places to support
Windows grammar. In the case of MSVC, tests can be run with this
command:
vcregress tapcheck
---
 doc/src/sgml/install-windows.sgml  |  1 +
 src/Makefile.global.in |  2 +-
 src/bin/initdb/t/001_initdb.pl |  1 +
 src/bin/pg_basebackup/t/010_pg_basebackup.pl   | 67 --
 src/bin/pg_controldata/t/001_pg_controldata.pl |  5 +-
 src/bin/pg_ctl/t/001_start_stop.pl | 14 --
 src/bin/pg_ctl/t/002_status.pl | 12 -
 src/bin/pg_rewind/RewindTest.pm| 66 +++--
 src/bin/scripts/t/020_createdb.pl  |  3 ++
 src/test/perl/TestLib.pm   | 18 ---
 src/tools/msvc/Solution.pm |  1 +
 src/tools/msvc/config_default.pl   |  1 +
 src/tools/msvc/vcregress.pl| 48 +-
 13 files changed, 175 insertions(+), 64 deletions(-)

diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windows.sgml
index d154b44..2047790 100644
--- a/doc/src/sgml/install-windows.sgml
+++ b/doc/src/sgml/install-windows.sgml
@@ -439,6 +439,7 @@ $ENV{CONFIG}=Debug;
 userinputvcregress modulescheck/userinput
 userinputvcregress ecpgcheck/userinput
 userinputvcregress isolationcheck/userinput
+userinputvcregress tapcheck/userinput
 userinputvcregress upgradecheck/userinput
 /screen
 
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index c583b44..563d1d1 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -336,7 +336,7 @@ cd $(srcdir)  TESTDIR='$(CURDIR)' PATH=$(bindir):$$PATH PGPORT='6$(DEF_PGPOR
 endef
 
 define prove_check
-cd $(srcdir)  TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir)  TESTDIR='$(CURDIR)' TESTREGRESS='$(top_builddir)/src/test/regress/pg_regress' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
 endef
 
 else
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index 299dcf5..095cbf3 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -4,6 +4,7 @@
 
 use strict;
 use warnings;
+use Config;
 use TestLib;
 use Test::More tests = 14;
 
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 3476ea6..112f3a1 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -1,8 +1,9 @@
 use strict;
 use warnings;
 use Cwd;
+use Config;
 use TestLib;
-use Test::More tests = 35;
+use Test::More tests = ($Config{osname} eq MSWin32) ? 25 : 35;
 
 program_help_ok('pg_basebackup');
 program_version_ok('pg_basebackup');
@@ -17,10 +18,20 @@ command_fails(
 	[ 'pg_basebackup', '-D', $tempdir/backup ],
 	'pg_basebackup fails because of hba');
 
+# Use SSPI on Windows, node has been initialized already accordingly
+# by pg_regress --config-auth.
 open HBA, $tempdir/pgdata/pg_hba.conf;
-print HBA local replication all trust\n;
-print HBA host replication all 127.0.0.1/32 trust\n;
-print HBA host replication all ::1/128 trust\n;
+if ($Config{osname} ne MSWin32)
+{
+	print HBA local replication all trust\n;
+	print HBA host replication all 127.0.0.1/32 trust\n;
+	print HBA host replication all ::1/128 trust\n;
+}
+else
+{
+	print HBA host replication all 127.0.0.1/32 sspi include_realm=1 map=regress\n;
+	print HBA host replication all ::1/128 sspi include_realm=1 map=regress\n;
+}
 close HBA;
 system_or_bail 'pg_ctl', '-s', '-D', $tempdir/pgdata, 'reload';
 
@@ -56,6 +67,33 @@ command_fails([ 'pg_basebackup', '-D', $tempdir/tarbackup_l1, '-Ft' ],
 	'pg_basebackup tar with long name fails');
 unlink $tempdir/pgdata/$superlongname;
 
+command_fails(
+	[ 'pg_basebackup', '-D', $tempdir/backup_foo, '-Fp', -T=/foo ],
+	'-T with

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-04-23 Thread Michael Paquier 
On Mon, Apr 20, 2015 at 9:01 PM, Michael Paquier
michael.paqu...@gmail.com wrote:
 On Sun, Apr 19, 2015 at 10:01 PM, Michael Paquier
 michael.paqu...@gmail.com wrote:
 Note as well that this patch uses the following patches fixing
 independent issues:
 ...

 Attached is v4. I added a switch in config.pl to be consistent with
 ./configure and --enable-tap-tests.

Attached is v5, rebased on HEAD (2c47fe16) after conflicts with
dcae5fac and 54a16df0.
-- 
Michael
From 9c348d985a3da0098be9d9f64a14e936068117a0 Mon Sep 17 00:00:00 2001
From: Michael Paquier mich...@otacoo.com
Date: Mon, 20 Apr 2015 04:57:37 -0700
Subject: [PATCH 1/2] Add support for TAP tests on Windows

Nodes initialized by the TAP tests use SSPI to securely perform the
tests, and test scripts are patched in a couple of places to support
Windows grammar. In the case of MSVC, tests can be run with this
command:
vcregress tapcheck
---
 doc/src/sgml/install-windows.sgml  |  1 +
 src/Makefile.global.in |  2 +-
 src/bin/initdb/t/001_initdb.pl | 18 --
 src/bin/pg_basebackup/t/010_pg_basebackup.pl   | 79 --
 src/bin/pg_controldata/t/001_pg_controldata.pl |  5 +-
 src/bin/pg_ctl/t/001_start_stop.pl | 14 -
 src/bin/pg_ctl/t/002_status.pl | 12 +++-
 src/bin/pg_rewind/RewindTest.pm| 56 ++
 src/bin/scripts/t/020_createdb.pl  |  3 +
 src/test/perl/TestLib.pm   | 16 --
 src/tools/msvc/Solution.pm |  1 +
 src/tools/msvc/config_default.pl   |  1 +
 src/tools/msvc/vcregress.pl| 48 +++-
 13 files changed, 195 insertions(+), 61 deletions(-)

diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windows.sgml
index d154b44..2047790 100644
--- a/doc/src/sgml/install-windows.sgml
+++ b/doc/src/sgml/install-windows.sgml
@@ -439,6 +439,7 @@ $ENV{CONFIG}=Debug;
 userinputvcregress modulescheck/userinput
 userinputvcregress ecpgcheck/userinput
 userinputvcregress isolationcheck/userinput
+userinputvcregress tapcheck/userinput
 userinputvcregress upgradecheck/userinput
 /screen
 
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 81cae94..f030dac 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -338,7 +338,7 @@ cd $(srcdir)  TESTDIR='$(CURDIR)' PATH=$(bindir):$$PATH PGPORT='6$(DEF_PGPOR
 endef
 
 define prove_check
-cd $(srcdir)  TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir)  TESTDIR='$(CURDIR)' TESTREGRESS='$(top_builddir)/src/test/regress/pg_regress' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
 endef
 
 else
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index d12be84..0865107 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -1,6 +1,8 @@
 use strict;
 use warnings;
+use Config;
 use TestLib;
+use File::Path qw(rmtree);
 use Test::More tests = 19;
 
 my $tempdir = TestLib::tempdir;
@@ -18,27 +20,31 @@ command_fails([ 'initdb', '-S', $tempdir/data3 ],
 mkdir $tempdir/data4 or BAIL_OUT($!);
 command_ok([ 'initdb', $tempdir/data4 ], 'existing empty data directory');
 
-system_or_bail rm -rf '$tempdir'/*;
-
+rmtree($tempdir);
+mkdir $tempdir;
 command_ok([ 'initdb', '-X', $tempdir/pgxlog, $tempdir/data ],
 	'separate xlog directory');
 
-system_or_bail rm -rf '$tempdir'/*;
+rmtree($tempdir);
+mkdir $tempdir;
 command_fails(
 	[ 'initdb', $tempdir/data, '-X', 'pgxlog' ],
 	'relative xlog directory not allowed');
 
-system_or_bail rm -rf '$tempdir'/*;
+rmtree($tempdir);
+mkdir $tempdir;
 mkdir $tempdir/pgxlog;
 command_ok([ 'initdb', '-X', $tempdir/pgxlog, $tempdir/data ],
 	'existing empty xlog directory');
 
-system_or_bail rm -rf '$tempdir'/*;
+rmtree($tempdir);
+mkdir $tempdir;
 mkdir $tempdir/pgxlog;
 mkdir $tempdir/pgxlog/lost+found;
 command_fails([ 'initdb', '-X', $tempdir/pgxlog, $tempdir/data ],
 	'existing nonempty xlog directory');
 
-system_or_bail rm -rf '$tempdir'/*;
+rmtree($tempdir);
+mkdir $tempdir;
 command_ok([ 'initdb', '-T', 'german', $tempdir/data ],
 	'select default dictionary');
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 7e9a776..4cb1b5a 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -1,8 +1,9 @@
 use strict;
 use warnings;
 use Cwd;
+use Config;
 use TestLib;
-use Test::More tests = 35;
+use Test::More tests = ($Config{osname} eq MSWin32) ? 26 : 35;
 
 program_help_ok('pg_basebackup');
 program_version_ok('pg_basebackup');
@@ -17,10 +18,20 @@ command_fails(
 	[ 'pg_basebackup', '-D', $tempdir/backup ],
 	'pg_basebackup fails because of hba');
 
+# Use SSPI on Windows, node has been initialized

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-04-20 Thread Michael Paquier 
On Sun, Apr 19, 2015 at 10:01 PM, Michael Paquier
michael.paqu...@gmail.com wrote:
 Note as well that this patch uses the following patches fixing
 independent issues:
 ...

Attached is v4. I added a switch in config.pl to be consistent with
./configure and --enable-tap-tests.
-- 
Michael
From f532bbd2522b3a1784038d9863325d055fc445bf Mon Sep 17 00:00:00 2001
From: Michael Paquier mich...@otacoo.com
Date: Mon, 20 Apr 2015 04:57:37 -0700
Subject: [PATCH 1/2] Add support for TAP tests on Windows

Nodes initialized by the TAP tests use SSPI to securely perform the
tests, and test scripts are patched in a couple of places to support
Windows grammar. In the case of MSVC, tests can be run with this
command:
vcregress tapcheck
---
 .gitignore |   3 +
 doc/src/sgml/install-windows.sgml  |   1 +
 src/Makefile.global.in |   2 +-
 src/bin/initdb/t/001_initdb.pl |  18 ++--
 src/bin/pg_basebackup/t/010_pg_basebackup.pl   |  79 +---
 src/bin/pg_controldata/t/001_pg_controldata.pl |   5 +-
 src/bin/pg_ctl/t/001_start_stop.pl |  14 ++-
 src/bin/pg_ctl/t/002_status.pl |  12 ++-
 src/bin/pg_rewind/RewindTest.pm| 119 -
 src/bin/scripts/t/020_createdb.pl  |   3 +
 src/test/perl/TestLib.pm   |  25 --
 src/tools/msvc/Solution.pm |   1 +
 src/tools/msvc/config_default.pl   |   1 +
 src/tools/msvc/vcregress.pl|  60 -
 14 files changed, 271 insertions(+), 72 deletions(-)

diff --git a/.gitignore b/.gitignore
index 8d3af50..3cd37fe 100644
--- a/.gitignore
+++ b/.gitignore
@@ -36,3 +36,6 @@ lib*.pc
 /pgsql.sln.cache
 /Debug/
 /Release/
+
+# Generated by tests
+/tmp_check/
diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windows.sgml
index d154b44..2047790 100644
--- a/doc/src/sgml/install-windows.sgml
+++ b/doc/src/sgml/install-windows.sgml
@@ -439,6 +439,7 @@ $ENV{CONFIG}=Debug;
 userinputvcregress modulescheck/userinput
 userinputvcregress ecpgcheck/userinput
 userinputvcregress isolationcheck/userinput
+userinputvcregress tapcheck/userinput
 userinputvcregress upgradecheck/userinput
 /screen
 
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 4b06fc2..b27ed78 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -324,7 +324,7 @@ endef
 define prove_check
 $(MKDIR_P) tmp_check/log
 $(MAKE) -C $(top_builddir) DESTDIR='$(CURDIR)'/tmp_check/install install '$(CURDIR)'/tmp_check/log/install.log 21
-cd $(srcdir)  TESTDIR='$(CURDIR)' PATH=$(CURDIR)/tmp_check/install$(bindir):$$PATH $(call add_to_path,$(ld_library_path_var),$(CURDIR)/tmp_check/install$(libdir)) top_builddir='$(CURDIR)/$(top_builddir)' PGPORT='6$(DEF_PGPORT)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir)  TESTDIR='$(CURDIR)' TESTREGRESS='$(top_builddir)/src/test/regress/pg_regress' PATH=$(CURDIR)/tmp_check/install$(bindir):$$PATH $(call add_to_path,$(ld_library_path_var),$(CURDIR)/tmp_check/install$(libdir)) top_builddir='$(CURDIR)/$(top_builddir)' PGPORT='6$(DEF_PGPORT)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
 endef
 
 else
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index d12be84..0865107 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -1,6 +1,8 @@
 use strict;
 use warnings;
+use Config;
 use TestLib;
+use File::Path qw(rmtree);
 use Test::More tests = 19;
 
 my $tempdir = TestLib::tempdir;
@@ -18,27 +20,31 @@ command_fails([ 'initdb', '-S', $tempdir/data3 ],
 mkdir $tempdir/data4 or BAIL_OUT($!);
 command_ok([ 'initdb', $tempdir/data4 ], 'existing empty data directory');
 
-system_or_bail rm -rf '$tempdir'/*;
-
+rmtree($tempdir);
+mkdir $tempdir;
 command_ok([ 'initdb', '-X', $tempdir/pgxlog, $tempdir/data ],
 	'separate xlog directory');
 
-system_or_bail rm -rf '$tempdir'/*;
+rmtree($tempdir);
+mkdir $tempdir;
 command_fails(
 	[ 'initdb', $tempdir/data, '-X', 'pgxlog' ],
 	'relative xlog directory not allowed');
 
-system_or_bail rm -rf '$tempdir'/*;
+rmtree($tempdir);
+mkdir $tempdir;
 mkdir $tempdir/pgxlog;
 command_ok([ 'initdb', '-X', $tempdir/pgxlog, $tempdir/data ],
 	'existing empty xlog directory');
 
-system_or_bail rm -rf '$tempdir'/*;
+rmtree($tempdir);
+mkdir $tempdir;
 mkdir $tempdir/pgxlog;
 mkdir $tempdir/pgxlog/lost+found;
 command_fails([ 'initdb', '-X', $tempdir/pgxlog, $tempdir/data ],
 	'existing nonempty xlog directory');
 
-system_or_bail rm -rf '$tempdir'/*;
+rmtree($tempdir);
+mkdir $tempdir;
 command_ok([ 'initdb', '-T', 'german', $tempdir/data ],
 	'select default dictionary');
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 7e9a776..4cb1b5a 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -1,8 +1,9 @@
 use

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-04-03 Thread Michael Paquier 
Thanks for your input, Noah.

On Fri, Apr 3, 2015 at 3:22 PM, Noah Misch n...@leadboat.com wrote:
 Each Windows patch should cover all Windows build systems; patches that fix a
 problem in the src/tools/msvc build while leaving it broken in the GNU make
 build are a bad trend.  In this case, I expect you'll need few additional
 changes to cover both.

Yes I am planning to do more tests with MinGW as well, once I got the
patch in a more advanced state in May/June. For Cygwin, I am not much
familiar with it yet, but I am sure I'll come up with something.

 On Thu, Apr 02, 2015 at 06:30:02PM +0900, Michael Paquier wrote:
 2) tapcheck does not check if IPC::Run is installed or not. Should we
 add a check in the code for that? If yes, should we bypass the test or
 fail?

 The src/tools/msvc build system officially requires ActivePerl, and I seem to
 recall ActivePerl installs IPC::Run by default.  A check is optional.

I recall installing ActivePerl for my own box some time ago. It is
5.16, so now it is outdated, but the package manager does not contain
IPC::Run and I had to install it by hand.

 Test suites that run as part of make check-world, including all src/bin TAP
 suites, _must not_ enable trust authentication on Windows.  To do so would
 reintroduce CVE-2014-0067.  (The standard alternative is to call pg_regress
 --config-auth, which switches a data directory to SSPI authentication.)
 Suites not in check-world, though not obligated to follow that rule, will have
 a brighter future if they do so anyway.

OK. I'll rework this part.

 --- a/src/test/perl/TestLib.pm
 +++ b/src/test/perl/TestLib.pm

 @@ -73,9 +74,19 @@ sub tempdir_short
  sub standard_initdb
  {
   my $pgdata = shift;
 - system_or_bail(initdb -D '$pgdata' -A trust -N /dev/null);
 - system_or_bail($ENV{top_builddir}/src/test/regress/pg_regress,
 -'--config-auth', $pgdata);
 +
 + if ($Config{osname} eq MSWin32)
 + {
 + system_or_bail(initdb -D $pgdata -A trust -N  NUL);
 + system_or_bail($ENV{TESTREGRESS},
 +'--config-auth', $pgdata);
 + }
 + else
 + {
 + system_or_bail(initdb -D '$pgdata' -A trust -N /dev/null);
 + 
 system_or_bail($ENV{top_builddir}/src/test/regress/pg_regress,
 +'--config-auth', $pgdata);
 + }
  }

 Make all build systems set TESTREGRESS, and use it unconditionally.

Yeah, that would be better.

 That's not a complete review, just some highlights.

Thanks again.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting TAP tests with MSVC and Windows

2015-04-03 Thread Noah Misch 
Each Windows patch should cover all Windows build systems; patches that fix a
problem in the src/tools/msvc build while leaving it broken in the GNU make
build are a bad trend.  In this case, I expect you'll need few additional
changes to cover both.

On Thu, Apr 02, 2015 at 06:30:02PM +0900, Michael Paquier wrote:
 2) tapcheck does not check if IPC::Run is installed or not. Should we
 add a check in the code for that? If yes, should we bypass the test or
 fail?

The src/tools/msvc build system officially requires ActivePerl, and I seem to
recall ActivePerl installs IPC::Run by default.  A check is optional.

 7) TAP tests use sometimes top_builddir directly. I think that's ugly,
 and if there are no objections I think that we should change it to use
 a dedicated environment variable like TESTTOP or similar.

I sense nothing ugly about a top_builddir reference, but see below.

 --- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
 +++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl

  open HBA, $tempdir/pgdata/pg_hba.conf;
 -print HBA local replication all trust\n;
 +# Windows builds do not support local connections
 +if ($Config{osname} ne MSWin32)
 +{
 + print HBA local replication all trust\n;
 +}
  print HBA host replication all 127.0.0.1/32 trust\n;
  print HBA host replication all ::1/128 trust\n;
  close HBA;

Test suites that run as part of make check-world, including all src/bin TAP
suites, _must not_ enable trust authentication on Windows.  To do so would
reintroduce CVE-2014-0067.  (The standard alternative is to call pg_regress
--config-auth, which switches a data directory to SSPI authentication.)
Suites not in check-world, though not obligated to follow that rule, will have
a brighter future if they do so anyway.

 --- a/src/test/perl/TestLib.pm
 +++ b/src/test/perl/TestLib.pm

 @@ -73,9 +74,19 @@ sub tempdir_short
  sub standard_initdb
  {
   my $pgdata = shift;
 - system_or_bail(initdb -D '$pgdata' -A trust -N /dev/null);
 - system_or_bail($ENV{top_builddir}/src/test/regress/pg_regress,
 -'--config-auth', $pgdata);
 +
 + if ($Config{osname} eq MSWin32)
 + {
 + system_or_bail(initdb -D $pgdata -A trust -N  NUL);
 + system_or_bail($ENV{TESTREGRESS},
 +'--config-auth', $pgdata);
 + }
 + else
 + {
 + system_or_bail(initdb -D '$pgdata' -A trust -N /dev/null);
 + system_or_bail($ENV{top_builddir}/src/test/regress/pg_regress,
 +'--config-auth', $pgdata);
 + }
  }

Make all build systems set TESTREGRESS, and use it unconditionally.


That's not a complete review, just some highlights.

Thanks,
nm


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Supporting TAP tests with MSVC and Windows

2015-04-02 Thread Michael Paquier 
Hi all,
(Adding Peter and Heikki in CC for awareness)

Please find attached a WIP patch to support TAP tests with MSVC. The
tests can be kicked with this command:
vcregress tapcheck

There are a couple of things to note with this patch, and I would like
to have some input for some of those things:
1) I have tweaked some code paths to configure a node correctly, with
for example listen_addresses = 'localhost', or disabling local
settings in pg_hba.conf. Does that sound fine?
2) tapcheck does not check if IPC::Run is installed or not. Should we
add a check in the code for that? If yes, should we bypass the test or
fail?
3) Temporary installation needs to be done in the top directory,
actually out of src/, because Install.pm scans the contents of src/ to
fetch for example the .samples files and this leads to bad
interactions. Using this location has as well the advantage to install
the binaries for all the tests only once.
4) pg_rewind tests are disabled for now, I am waiting for the outcome
of 5519f169.8030...@gmx.net
5) ssl tests are disabled for now (haven't looked at that yet). They
should be tested if possible. Still need some investigation.
6) the command to access pg_regress is passed using an environment
variable TESTREGRESS.
7) TAP tests use sometimes top_builddir directly. I think that's ugly,
and if there are no objections I think that we should change it to use
a dedicated environment variable like TESTTOP or similar.
8) Some commands have needed some tweaks because option parsing on
Windows is... Well... sometimes broken. For example those things do
not work on Windows:
createdb foobar3 -E win1252
initdb PGDATA -X XLOGDIR
Note that modifying those commands does not change the test coverage.
9) @Peter: IPC::Run is not reliable when using h-results, by
switching to (h-full_results)[0] I was able to get results reliably
from a child process.
10) This patch needs to have IPC::Run installed. You need to install
it from CPAN, that's the same deal as for OSX.
11) Windows does not support symlink, so some tests of pg_basebackup
cannot be executed. I can live with this restriction.

I think that's all for now. I'll add this patch to the 2015-06 commit fest.
Regards,
-- 
Michael
diff --git a/.gitignore b/.gitignore
index 8d3af50..3cd37fe 100644
--- a/.gitignore
+++ b/.gitignore
@@ -36,3 +36,6 @@ lib*.pc
 /pgsql.sln.cache
 /Debug/
 /Release/
+
+# Generated by tests
+/tmp_check/
diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windows.sgml
index 9b77648..d3d8f5f 100644
--- a/doc/src/sgml/install-windows.sgml
+++ b/doc/src/sgml/install-windows.sgml
@@ -438,6 +438,7 @@ $ENV{CONFIG}=Debug;
 userinputvcregress contribcheck/userinput
 userinputvcregress ecpgcheck/userinput
 userinputvcregress isolationcheck/userinput
+userinputvcregress tapcheck/userinput
 userinputvcregress upgradecheck/userinput
 /screen
 
diff --git a/src/bin/initdb/t/001_initdb.pl b/src/bin/initdb/t/001_initdb.pl
index 149b3d1..091ec0e 100644
--- a/src/bin/initdb/t/001_initdb.pl
+++ b/src/bin/initdb/t/001_initdb.pl
@@ -1,10 +1,26 @@
 use strict;
 use warnings;
+use Config;
 use TestLib;
 use Test::More tests = 19;
 
 my $tempdir = TestLib::tempdir;
 
+sub cleanup_tempdir
+{
+	my $tempdir = shift;
+
+	if ($Config{osname} eq MSWin32)
+	{
+		system_or_bail rd /s /q $tempdir;
+		mkdir $tempdir;
+	}
+	else
+	{
+		system_or_bail rm -rf '$tempdir'/*;
+	}
+}
+
 program_help_ok('initdb');
 program_version_ok('initdb');
 program_options_handling_ok('initdb');
@@ -18,27 +34,26 @@ command_fails([ 'initdb', '-S', $tempdir/data3 ],
 mkdir $tempdir/data4 or BAIL_OUT($!);
 command_ok([ 'initdb', $tempdir/data4 ], 'existing empty data directory');
 
-system_or_bail rm -rf '$tempdir'/*;
-
-command_ok([ 'initdb', $tempdir/data, '-X', $tempdir/pgxlog ],
-	'separate xlog directory');
+cleanup_tempdir $tempdir;
 
-system_or_bail rm -rf '$tempdir'/*;
+command_ok([ 'initdb', '-D', $tempdir/data, '-X', $tempdir/pgxlog ],
+		   'separate xlog directory');
+cleanup_tempdir $tempdir;
 command_fails(
 	[ 'initdb', $tempdir/data, '-X', 'pgxlog' ],
 	'relative xlog directory not allowed');
 
-system_or_bail rm -rf '$tempdir'/*;
+cleanup_tempdir $tempdir;
 mkdir $tempdir/pgxlog;
-command_ok([ 'initdb', $tempdir/data, '-X', $tempdir/pgxlog ],
+command_ok([ 'initdb', '-D', $tempdir/data, '-X', $tempdir/pgxlog ],
 	'existing empty xlog directory');
 
-system_or_bail rm -rf '$tempdir'/*;
+cleanup_tempdir $tempdir;
 mkdir $tempdir/pgxlog;
 mkdir $tempdir/pgxlog/lost+found;
-command_fails([ 'initdb', $tempdir/data, '-X', $tempdir/pgxlog ],
+command_fails([ 'initdb', '-D', $tempdir/data, '-X', $tempdir/pgxlog ],
 	'existing nonempty xlog directory');
 
-system_or_bail rm -rf '$tempdir'/*;
-command_ok([ 'initdb', $tempdir/data, '-T', 'german' ],
+cleanup_tempdir $tempdir;
+command_ok([ 'initdb', '-D', $tempdir/data, '-T', 'german' ],
 	'select default dictionary');
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl