Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-24 Thread Alvaro Herrera
On 2019-Jan-24, Tom Lane wrote: > > Also, as > > the pseudo-random state is fully controlled, seeded test results are > > deterministic so the expected value can be fully checked. > > I found that the "expected value" was different in v11 than HEAD, > which surprised me. It looks like the

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-24 Thread Tom Lane
Fabien COELHO writes: >> I had in mind something more like the attached. > Yep. > I'm not too happy that it mixes API levels, and about the int/double/int > path. > Attached an updated version which relies on pg_jrand48 instead. Hm, I'm not sure that's really an improvement, but I pushed it

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-22 Thread Fabien COELHO
Hello Tom, Here is a POC which defines an internal interface for a PRNG, and use it within pgbench, with several possible implementations which default to rand48. I seriously dislike this patch. pgbench's random support is quite overengineered already IMO, and this proposes to add a whole

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-22 Thread Fabien COELHO
It's not demonstrably slower than 2.5 either. (1.1 is measurably slower; probably not by enough for anyone to care, but 1.5 is good enough for me.) Good if it fails quick enough for you. Attached a patch with the zipf doc update & the TAP test parameter change. -- Fabien.diff --git

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-22 Thread Fabien COELHO
The first value is taken about 75% of the time for N=1000 and s=2.5, which means that a non deterministic implementation would succeed about 0.75² ~ 56% of the time on that one. Right, that's about what we've been seeing on OpenBSD. Also, the drawing procedure is less efficient when the

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-22 Thread Tom Lane
Fabien COELHO writes: >> I'm not following this argument. The test case is basically useless >> for its intended purpose with that parameter, because it's highly >> likely that the failure mode it's supposedly checking for will be >> masked by the "random" function's tendency to spit out the

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-22 Thread Fabien COELHO
Hello Tom, BTW, did you look at the question of the range of zipfian? Yep. I confirmed here that as used in the test case, it's generating a range way smaller than the other ones: repeating the insertion snippet 1000x produces stats like this: [...] I have no idea whether that

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-20 Thread Fabien COELHO
Hello Tom, Here is a POC which defines an internal interface for a PRNG, and use it within pgbench, with several possible implementations which default to rand48. I seriously dislike this patch. pgbench's random support is quite overengineered already IMO, and this proposes to add a whole

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-20 Thread Tom Lane
Fabien COELHO writes: >>> I'd suggest that maybe we should get rid of the use of both random() >>> and srandom() in pgbench, and go over to letting set_random_seed() >>> fill the pg_erand48 state directly. > Here is a POC which defines an internal interface for a PRNG, and use it > within

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-20 Thread Fabien COELHO
Hello Tom, Maybe on OpenBSD pg should switch srandom to srandom_deterministic? Dunno. I'm fairly annoyed by their idea that they're smarter than POSIX. Hmmm. I'm afraid that is not that hard. However, for most of our uses of srandom, this behavior isn't awful; it's only pgbench that has

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-18 Thread Fabien COELHO
Maybe on OpenBSD pg should switch srandom to srandom_deterministic? Dunno. I'm fairly annoyed by their idea that they're smarter than POSIX. However, for most of our uses of srandom, this behavior isn't awful; it's only pgbench that has an expectation that the platform random() can be made

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-18 Thread Tom Lane
Fabien COELHO writes: >> all is explained here: >> https://man.openbsd.org/srandom >> Or at least most is explained there. > Yep. They try to be more serious than other systems about PRNG, which is > not bad in itself. > Maybe on OpenBSD pg should switch srandom to srandom_deterministic?

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-18 Thread Fabien COELHO
BTW, if you're wondering why curculio is still failing the pgbench test, Hmmm, that is interesting! It shows that at least some TAP tests are useful. all is explained here: https://man.openbsd.org/srandom Or at least most is explained there. Yep. They try to be more serious than

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-18 Thread Fabien COELHO
I'd rather keep it by simply adding the "|unknown" alternative. 30 years of programming have taught me that testing limit & error cases is useful, although you never know when it will be proven so. Sorry, I don't buy this line of argument. Reasonable test design requires making

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Tom Lane
BTW, if you're wondering why curculio is still failing the pgbench test, all is explained here: https://man.openbsd.org/srandom Or at least most is explained there. While curculio is unsurprisingly failing all four seeded_random tests, when I try it locally on an OpenBSD 6.4 installation, only

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Michael Paquier
On Thu, Jan 17, 2019 at 07:21:08PM -0500, Tom Lane wrote: > Sorry, I don't buy this line of argument. Reasonable test design requires > making cost/benefit tradeoffs: the cost to run the test over and over, > and the cost to maintain the test itself (e.g. fix portability issues in > it) have to

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Tom Lane
Fabien COELHO writes: >> I am, TBH, inclined to fix this by removing that test case rather >> than teaching it another spelling to accept. I think it's very >> hard to make the case that tests like this one are anything but >> a waste of developer and buildfarm time. When they are also a >>

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-18 00:31, Tom Lane wrote: =?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: And now also the NetBSD machine failed on pgbenchCheck. Indeed, as expected. Ok. should I leave it as it is for now? Please. I'll push a fix for the broken test case in a bit --- I just wanted to

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Tom Lane
=?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: > And now also the NetBSD machine failed on pgbenchCheck. Indeed, as expected. > should I leave it as it is for now? Please. I'll push a fix for the broken test case in a bit --- I just wanted to confirm that somebody else's machines agreed that it's

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Tom Lane
=?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: >> But it looks like in NetBSD the options are called: Sorry about that, I copied-and-pasted from the openbsd machine I was looking at without remembering that netbsd is just a shade different. > but the OpenBSD machine went further and now fails on: >

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-18 00:00, Mikael Kjellström wrote: I just started another run on sidewinder (NetBSD 7), let's see how that goes. but the OpenBSD machine went further and now fails on: pgbenchCheck instead. Is that the failure you expected to get? And now also the NetBSD machine failed on

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-17 23:54, Mikael Kjellström wrote: But it looks like in NetBSD the options are called: netbsd7-pgbf# sysctl -a | grep semmn kern.ipc.semmni = 10 kern.ipc.semmns = 60 kern.ipc.semmnu = 30 so I will try and set that in /etc/sysctl.conf and reboot and see what happens. That

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-17 23:37, Mikael Kjellström wrote: On 2019-01-17 23:23, Tom Lane wrote: Yeah, you might've been able to get by with OpenBSD/NetBSD's default semaphore settings before, but they really only let one postmaster run at a time; and the TAP tests want to start more than one. For me it

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-17 23:23, Tom Lane wrote: Yeah, you might've been able to get by with OpenBSD/NetBSD's default semaphore settings before, but they really only let one postmaster run at a time; and the TAP tests want to start more than one. For me it seems to work to append this to

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Tom Lane
=?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: >> Let's see if it works better this time. > Hmmm, nope: > 2019-01-17 23:09:20.343 CET [9129] FATAL: could not create semaphores: > No space left on device Yeah, you might've been able to get by with OpenBSD/NetBSD's default semaphore settings

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-17 22:47, Mikael Kjellström wrote: On 2019-01-17 22:42, Tom Lane wrote: =?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: It says: configure: error: Additional Perl modules are required to run TAP tests so how do I find out with Perl modules that are required? If you look into the

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-17 22:42, Tom Lane wrote: =?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: It says: configure: error: Additional Perl modules are required to run TAP tests so how do I find out with Perl modules that are required? If you look into the configure log it should say just above that, but

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Tom Lane
=?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: > It says: > configure: error: Additional Perl modules are required to run TAP tests > so how do I find out with Perl modules that are required? If you look into the configure log it should say just above that, but I'm betting you just need p5-IPC-Run.

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-17 22:19, Mikael Kjellström wrote: On 2019-01-17 22:16, Tom Lane wrote: For what it's worth I've enabled tap-tests for my OpenBSD 5.9 (curculio) and NetBSD 7 (sidewinder) animals now. Oh, thanks!  I'm guessing they'll fail their next runs, but I'll wait to see confirmation of

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-17 22:16, Tom Lane wrote: For what it's worth I've enabled tap-tests for my OpenBSD 5.9 (curculio) and NetBSD 7 (sidewinder) animals now. Oh, thanks! I'm guessing they'll fail their next runs, but I'll wait to see confirmation of that before I do anything about the test bug.

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Tom Lane
=?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: > On 2019-01-17 06:04, Tom Lane wrote: >> Although we've got a few NetBSD and OpenBSD buildfarm critters, >> none of them are doing --enable-tap-tests. If they were, we'd >> have noticed the pgbench regression tests falling over: > For what it's worth

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Mikael Kjellström
On 2019-01-17 06:04, Tom Lane wrote: Although we've got a few NetBSD and OpenBSD buildfarm critters, none of them are doing --enable-tap-tests. If they were, we'd have noticed the pgbench regression tests falling over: For what it's worth I've enabled tap-tests for my OpenBSD 5.9 (curculio)

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-17 Thread Fabien COELHO
Hello Tom, Although we've got a few NetBSD and OpenBSD buildfarm critters, none of them are doing --enable-tap-tests. If they were, we'd have noticed the pgbench regression tests falling over: [...] I am, TBH, inclined to fix this by removing that test case rather than teaching it another

PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-16 Thread Tom Lane
Although we've got a few NetBSD and OpenBSD buildfarm critters, none of them are doing --enable-tap-tests. If they were, we'd have noticed the pgbench regression tests falling over: not ok 3 - pgbench option error: bad option stderr /(?^:(unrecognized|illegal) option)/ # Failed test 'pgbench