Experimenting with hash join prefetch

2018-10-13 Thread Thomas Munro
Hello hackers, I have a list of micro-optimisations and things to look into for hash joins, which I've updated on the wiki[1]. Here's one that I was inspired to poke at with a stick in a spare hour today. Cache-oblivious hash joins cause a lot of TLB and cache misses. Researchers tell us to

Re: pgsql: Add TAP tests for pg_verify_checksums

2018-10-13 Thread Michael Paquier
On Sat, Oct 13, 2018 at 05:53:00PM -0400, Andrew Dunstan wrote: > It occurred to me that a pretty simple fix could just be to blacklist > everything that didn't start with a digit. The whitelist approach is > probably preferable... depends how urgent we see this as. Yeah, possibly, still that's

Re: fine tune v11 release notes

2018-10-13 Thread Tom Lane
Justin Pryzby writes: > On Sat, Oct 13, 2018 at 05:31:46PM -0400, Tom Lane wrote: >> I've pushed updates merging Jonathan's last with my work. Please >> look over what's in HEAD and see if you want to propose additional >> changes. > Find attached what's left. I pushed some of this verbatim,

Re: fine tune v11 release notes

2018-10-13 Thread Justin Pryzby
On Sat, Oct 13, 2018 at 05:31:46PM -0400, Tom Lane wrote: > Justin Pryzby writes: > >> On Sat, Oct 13, 2018 at 04:34:07PM -0400, Tom Lane wrote: > >>> Hmm, I just saw this after spending an afternoon editing the release > >>> notes. I'll try to merge it with what I did. > > > Feel free to send

Re: Regarding query minimizer (simplifier)

2018-10-13 Thread Tom Lane
"Jung, Jinho" writes: > Hello, I am Jinho Jung, Phd Student from GeorgiaTech, and try to find any SQL > queries that cause performance regression. While conducting evaluation, I > found an interesting query which makes x80 times slower execution in version > 10.5 than version 9.4. Please see

Re: pgsql: Add TAP tests for pg_verify_checksums

2018-10-13 Thread Andrew Dunstan
On 10/13/2018 10:00 AM, Andrew Dunstan wrote: On 10/13/2018 04:30 AM, Michael Paquier wrote: On Fri, Oct 12, 2018 at 12:14:48PM +0200, Michael Banck wrote: On Fri, Oct 12, 2018 at 03:05:43PM +0900, Michael Paquier wrote: On Fri, Oct 12, 2018 at 12:11:58PM +0900, Michael Paquier wrote:

Re: backpatch to v11? Add "B" suffix for bytes to docs

2018-10-13 Thread Tom Lane
Justin Pryzby writes: > I suggest this should be backpatched ? Yeah, done. regards, tom lane

Re: fine tune v11 release notes

2018-10-13 Thread Tom Lane
Justin Pryzby writes: >> On Sat, Oct 13, 2018 at 04:34:07PM -0400, Tom Lane wrote: >>> Hmm, I just saw this after spending an afternoon editing the release >>> notes. I'll try to merge it with what I did. > Feel free to send your own patch and I'll resolve conflicts and resend > proposed >

Regarding query minimizer (simplifier)

2018-10-13 Thread Jung, Jinho
Hello, I am Jinho Jung, Phd Student from GeorgiaTech, and try to find any SQL queries that cause performance regression. While conducting evaluation, I found an interesting query which makes x80 times slower execution in version 10.5 than version 9.4. Please see the attached files, if you are

backpatch to v11? Add "B" suffix for bytes to docs

2018-10-13 Thread Justin Pryzby
I suggest this should be backpatched ? commit 36e9d413a1d6928cd809d302d495fd6880a44b1e Author: Greg Stark Date: Sat Oct 6 13:03:43 2018 -0400 Add "B" suffix for bytes to docs 6e7baa3227 and b06d8e58b5 added "B" as a valid suffix for GUC_UNIT_BYTES but neglected to add it to

Re: fine tune v11 release notes

2018-10-13 Thread Justin Pryzby
On Sat, Oct 13, 2018 at 03:38:39PM -0500, Justin Pryzby wrote: > On Sat, Oct 13, 2018 at 04:34:07PM -0400, Tom Lane wrote: > > "Jonathan S. Katz" writes: > > > On 10/6/18 9:42 AM, Justin Pryzby wrote: > > >> Find below various fixes to release notes for v11, for discussion > > >> purposes. > >

Re: fine tune v11 release notes

2018-10-13 Thread Jonathan S. Katz
On 10/13/18 4:38 PM, Justin Pryzby wrote: > On Sat, Oct 13, 2018 at 04:34:07PM -0400, Tom Lane wrote: >> "Jonathan S. Katz" writes: >>> On 10/6/18 9:42 AM, Justin Pryzby wrote: Find below various fixes to release notes for v11, for discussion purposes. >> >>> Thanks for putting this

Re: fine tune v11 release notes

2018-10-13 Thread Justin Pryzby
On Sat, Oct 13, 2018 at 04:34:07PM -0400, Tom Lane wrote: > "Jonathan S. Katz" writes: > > On 10/6/18 9:42 AM, Justin Pryzby wrote: > >> Find below various fixes to release notes for v11, for discussion purposes. > > > Thanks for putting this together. I reviewed, broke it up into two > >

Re: fine tune v11 release notes

2018-10-13 Thread Tom Lane
"Jonathan S. Katz" writes: > On 10/6/18 9:42 AM, Justin Pryzby wrote: >> Find below various fixes to release notes for v11, for discussion purposes. > Thanks for putting this together. I reviewed, broke it up into two > patches, and made some additions / changes / deletions. Please see >

Re: fine tune v11 release notes

2018-10-13 Thread Jonathan S. Katz
Hi Justin, On 10/6/18 9:42 AM, Justin Pryzby wrote: > Find below various fixes to release notes for v11, for discussion purposes. Thanks for putting this together. I reviewed, broke it up into two patches, and made some additions / changes / deletions. Please see attached patches. Thanks!

Re: inconsistency and inefficiency in setup_conversion()

2018-10-13 Thread John Naylor
Attached is v6, a simple rebase. -John Naylor From e37cb80ab8e7baaa5231fc3b8dbc9d96ec253018 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Sat, 13 Oct 2018 19:28:08 +0700 Subject: [PATCH v6 1/2] Add pg_language lookup. This didn't seem worth doing before, but an upcoming commit will add 88

Re: WIP: Avoid creation of the free space map for small tables

2018-10-13 Thread John Naylor
On 10/13/18, Amit Kapila wrote: > On Sat, Oct 6, 2018 at 12:17 AM John Naylor wrote: >> -For normal mode, I taught fsm_set_and_search() to switch to a >> non-extending buffer call, but the biggest missing piece is WAL >> replay. >> > > fsm_set_and_search() > { > .. > + /* > + * For heaps we

RE: pgbench - doCustom cleanup

2018-10-13 Thread Fabien COELHO
Hello Kirk, I have decided to take a look into this patch. Thanks for the feedback. I noticed in your refactored code that CSTATE_CHOOSE_SCRIPT may now also change to CSTATE_FINISHED when timer is exceeded. (Before, it only changes to either CSTATE_START_THROTTLE or CSTATE_START_TX) But

Re: pg_dumpall --exclude-database option

2018-10-13 Thread Fabien COELHO
Hello Andrew, A question: would it makes sense to have a symmetrical --include-database=PATTERN option as well? I don't think so. If you only want a few databases, just use pg_dump. The premise of pg_dumpall is that you want all of them and this switch provides for exceptions to that.

Re: pgsql: Add TAP tests for pg_verify_checksums

2018-10-13 Thread Andrew Dunstan
On 10/13/2018 04:30 AM, Michael Paquier wrote: On Fri, Oct 12, 2018 at 12:14:48PM +0200, Michael Banck wrote: On Fri, Oct 12, 2018 at 03:05:43PM +0900, Michael Paquier wrote: On Fri, Oct 12, 2018 at 12:11:58PM +0900, Michael Paquier wrote: Agreed. I am just working on a patch for v11-

Re: WIP: Avoid creation of the free space map for small tables

2018-10-13 Thread Amit Kapila
On Sat, Oct 6, 2018 at 12:17 AM John Naylor wrote: > > Hi all, > A while back, Robert Haas noticed that the space taken up by very > small tables is dominated by the FSM [1]. Tom suggested that we could > prevent creation of the FSM until the heap has reached a certain > threshold size [2].

Re: Maximum password length

2018-10-13 Thread Alexander Kukushkin
On Sat, 13 Oct 2018 at 02:02, Tom Lane wrote: > Sure, but even a generated security token seems unlikely to be more > than a couple dozen bytes long. What's the actual use-case for tokens > longer than that? ISTM that a limit around 100 bytes already has a > whole lot of headroom.

Re: DSM segment handle generation in background workers

2018-10-13 Thread Thomas Munro
On Thu, Oct 11, 2018 at 5:51 PM Tom Lane wrote: > The comment adjacent to the change in InitStandaloneProcess bothers me. > In particular, it points out that what BackendRun() is currently doing > creates more entropy in the process's random seed than what you have > here: MyStartTime is only

Re: pgsql: Add TAP tests for pg_verify_checksums

2018-10-13 Thread Michael Paquier
On Fri, Oct 12, 2018 at 12:14:48PM +0200, Michael Banck wrote: > On Fri, Oct 12, 2018 at 03:05:43PM +0900, Michael Paquier wrote: >> On Fri, Oct 12, 2018 at 12:11:58PM +0900, Michael Paquier wrote: >>> Agreed. I am just working on a patch for v11- which uses a >>> whitelist-based method instead