[HACKERS] Size vs size_t

2017-03-16 Thread Thomas Munro
Hi, Noticing that the assembled hackers don't seem to agree on $SUBJECT in new patches, I decided to plot counts of lines matching \ and \ over time. After a very long run in the lead, size_t has recently been left in the dust by Size. -- Thomas Munro http://www.enterprisedb.com -- Sent via p

Re: [HACKERS] GSOC - TOAST'ing in slices

2017-03-16 Thread Stephen Frost
George, * George Papadrosou (gpapadro...@gmail.com) wrote: > Stephen, you mentioned PostGIS, but the conversation seems to lean towards > JSONB. What are your thoughts? Both are important. I brought up PostGIS specifically because it's an external project which could benefit from this work and

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-16 Thread Corey Huinker
On Mon, Mar 13, 2017 at 5:21 PM, Tom Lane wrote: > "Daniel Verite" writes: > > Tom Lane wrote: > >> when we see \if is that we do nothing but absorb text > >> until we see the matching \endif. At that point we could bitch and > throw > >> everything away if, say, there's \elif after \else, or a

Re: [HACKERS] Size vs size_t

2017-03-16 Thread Robert Haas
On Thu, Mar 16, 2017 at 4:40 PM, Thomas Munro wrote: > Noticing that the assembled hackers don't seem to agree on $SUBJECT in > new patches, I decided to plot counts of lines matching \ and > \ over time. After a very long run in the lead, size_t has > recently been left in the dust by Size. I g

Re: [HACKERS] Size vs size_t

2017-03-16 Thread Andres Freund
On 2017-03-16 16:59:29 -0400, Robert Haas wrote: > On Thu, Mar 16, 2017 at 4:40 PM, Thomas Munro > wrote: > > Noticing that the assembled hackers don't seem to agree on $SUBJECT in > > new patches, I decided to plot counts of lines matching \ and > > \ over time. After a very long run in the lead

[HACKERS] Re: new set of psql patches for loading (saving) data from (to) text, binary files

2017-03-16 Thread Stephen Frost
Pavel, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > 2017-03-15 17:21 GMT+01:00 Stephen Frost : > > I started looking through this to see if it might be ready to commit and > > I don't believe it is. Below are my comments about the first patch, I > > didn't get to the point of looking at the

Re: [HACKERS] Size vs size_t

2017-03-16 Thread Robert Haas
On Thu, Mar 16, 2017 at 5:01 PM, Andres Freund wrote: > On 2017-03-16 16:59:29 -0400, Robert Haas wrote: >> On Thu, Mar 16, 2017 at 4:40 PM, Thomas Munro >> wrote: >> > Noticing that the assembled hackers don't seem to agree on $SUBJECT in >> > new patches, I decided to plot counts of lines match

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-16 Thread Tom Lane
Corey Huinker writes: > Ok, I've got some time now and I'm starting to dig into this. I'd like to > restate what I *think* my feedback is, in case I missed or misunderstood > something. > ... > 3. Change command scans to scan the whole boolean expression, not just > OT_NORMAL. > There's a couple w

Re: [HACKERS] asynchronous execution

2017-03-16 Thread Corey Huinker
On Thu, Mar 16, 2017 at 4:17 PM, Tom Lane wrote: > Corey Huinker writes: > > I reworked the test such that all of the foreign tables inherit from the > > same parent table, and if you query that you do get async execution. But > It > > doesn't work when just stringing together those foreign tabl

Re: [HACKERS] Size vs size_t

2017-03-16 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 16, 2017 at 5:01 PM, Andres Freund wrote: >> On 2017-03-16 16:59:29 -0400, Robert Haas wrote: >>> I guess I assumed that we wouldn't have defined PG-specific types if >>> we wanted to just use the OS-supplied ones. >> I think, in this case, defining Size in the

[HACKERS] Re: [BUGS] Problem in using pgbench's --connect(-C) and --rate=rate(-R rate) options together.

2017-03-16 Thread David Steele
On 1/25/17 2:58 PM, Fabien COELHO wrote: > > Repost from bugs. This patch does not apply at cccbdde: $ patch -p1 < ../other/pgbench-CR-bug-1.patch (Stripping trailing CRs from patch.) patching file src/bin/pgbench/pgbench.c Hunk #1 FAILED at 1967. Hunk #2 succeeded at 4180 with fuzz 2 (offset -1

Re: [HACKERS] PL/Python: Add cursor and execute methods to plan object

2017-03-16 Thread David Steele
On 2/25/17 1:27 PM, Peter Eisentraut wrote: > Something that has been bothering me in PL/Python for a long time is the > non-object-oriented way in which plans are prepared and executed: > > plan = plpy.prepare(...) > res = plpy.execute(plan, ...) > > where plpy.execute() takes either a p

Re: [HACKERS] Size vs size_t

2017-03-16 Thread Andres Freund
On 2017-03-16 17:24:17 -0400, Tom Lane wrote: > Robert Haas writes: > > On Thu, Mar 16, 2017 at 5:01 PM, Andres Freund wrote: > >> On 2017-03-16 16:59:29 -0400, Robert Haas wrote: > >>> I guess I assumed that we wouldn't have defined PG-specific types if > >>> we wanted to just use the OS-supplie

Re: [HACKERS] Making clausesel.c Smarter

2017-03-16 Thread David Steele
On 2/26/17 1:41 AM, Robert Haas wrote: > On Fri, Feb 24, 2017 at 3:30 PM, David Rowley > wrote: >> It would be good to improve the situation here in the back branches >> too, but I'm thinking that the attached might be a little invasive for >> that? > > My experience has been that customers - at

Re: [HACKERS] Making clausesel.c Smarter

2017-03-16 Thread Tom Lane
David Steele writes: > Anyone familiar with the planner available to review this patch? FWIW, it's on my radar but I don't expect to get to it real soon, as there's other stuff I deem higher priority. In the meantime, I don't want to stand in the way of someone else looking at it.

Re: [HACKERS] Size vs size_t

2017-03-16 Thread Tom Lane
Andres Freund writes: > On 2017-03-16 17:24:17 -0400, Tom Lane wrote: >> The short answer to that is that "Size" predates the universal acceptance >> of size_t. If we were making these decisions today, or anytime since the >> early 2000s, we'd surely have just gone with size_t. But it wasn't a >

Re: [HACKERS] Size vs size_t

2017-03-16 Thread Thomas Munro
On Fri, Mar 17, 2017 at 10:39 AM, Andres Freund wrote: > On 2017-03-16 17:24:17 -0400, Tom Lane wrote: >> Robert Haas writes: >> > On Thu, Mar 16, 2017 at 5:01 PM, Andres Freund wrote: >> >> On 2017-03-16 16:59:29 -0400, Robert Haas wrote: >> > Well, I don't think we want to end up with a mix of

Re: [HACKERS] Size vs size_t

2017-03-16 Thread Tom Lane
Thomas Munro writes: > Naive replacement in new files (present in master but not in 9.6) with > the attached script, followed by a couple of manual corrections where > Size was really an English word in a comment, gets the attached diff. In the case of mmgr/slab.c, a lot of those uses of Size pro

Re: [HACKERS] BUG #13755: pgwin32_is_service not checking if SECURITY_SERVICE_SID is disabled

2017-03-16 Thread MauMau
From: Heikki Linnakangas So, I think we still need the check for Local System. Thanks, fixed and confirmed that the error message is output in the event log. Regards MauMau win32-security-service-v7.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

[HACKERS] [PATCH] Remove defunct and unnecessary link

2017-03-16 Thread David Christensen
The HA docs reference a “glossary” link which is no longer accessible, nor is it likely to be useful in general to link off-site IMHO. This simple patch removes this link. Best, David -- David Christensen End Point Corporation da...@endpoint.com 785-727-1171 0001-Remove-defunct-and-unneces

Re: [HACKERS] \h tab-completion

2017-03-16 Thread Peter Eisentraut
On 3/15/17 22:46, Andreas Karlsson wrote: > On 03/01/2017 02:47 PM, Peter Eisentraut wrote: >> Instead of creating another copy of list_ALTER, let's use the >> words_after_create list and write a version of >> create_command_generator/drop_command_generator. > > Good idea. Here is a patch with tha

Re: [HACKERS] \h tab-completion

2017-03-16 Thread Andreas Karlsson
On 03/17/2017 12:01 AM, Peter Eisentraut wrote: Committed with some tweaking. Thanks! Andreas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-16 Thread Stas Kelvich
>> On 2 Mar 2017, at 11:00, Craig Ringer wrote: >> >> BTW, I've been reviewing the patch in more detail. Other than a bunch >> of copy-and-paste that I'm cleaning up, the main issue I've found is >> that in DecodePrepare, you call: >> >> SnapBuildCommitTxn(ctx->snapshot_builder, buf->origptr,

Re: [HACKERS] Crash on promotion when recovery.conf is renamed

2017-03-16 Thread Tsunakawa, Takayuki
From: David Steele [mailto:da...@pgmasters.net] > Any idea when you'll have a chance to review? I'll do it by early next week. Regards Takayuki Tsunakawa -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpr

Re: [HACKERS] increasing the default WAL segment size

2017-03-16 Thread Robert Haas
On Tue, Mar 14, 2017 at 1:44 AM, Beena Emerson wrote: > Attached is the updated patch. It fixes the issues and also updates few code > comments. I did an initial readthrough of this patch tonight just to get a feeling for what's going on. Based on that, here are a few review comments: The chang

Re: [HACKERS] CREATE/ALTER ROLE PASSWORD ('value' USING 'method')

2017-03-16 Thread Michael Paquier
On Fri, Mar 17, 2017 at 2:30 AM, Jeff Janes wrote: > On Thu, Mar 9, 2017 at 4:59 AM, Michael Paquier > wrote: >> >> On Thu, Mar 9, 2017 at 1:17 AM, Joe Conway wrote: >> > On 03/07/2017 08:29 PM, Tom Lane wrote: >> >> Michael Paquier writes: >> >>> here is a separate thread dedicated to the foll

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-16 Thread Corey Huinker
Attached is the latest work. Not everything is done yet. I post it because the next step is likely to be "tedious" as Tom put it, and if there's a way out of it, I want to avoid it. What is done: - all changes here built off the v22 patch - any function which had scan_state and cond_stack passed i

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2017-03-16 Thread Michael Paquier
On Fri, Mar 17, 2017 at 12:19 AM, David Steele wrote: > This patch does not apply cleanly at cccbdde: > > $ git apply ../other/0001-Rename-pg_clog-to-pg_xact.patch > error: doc/src/sgml/ref/pg_resetxlog.sgml: No such file or directory > error: patch failed: src/backend/postmaster/autovacuum.c:2468

Re: [HACKERS] ANALYZE command progress checker

2017-03-16 Thread Robert Haas
On Fri, Mar 10, 2017 at 2:46 AM, vinayak wrote: > Thank you for reviewing the patch. > > The attached patch incorporated Michael and Amit comments also. I reviewed this tonight. +/* Report compute index stats phase */ +pgstat_progress_update_param(PROGRESS_ANALYZE_PHASE, + PROGRE

Re: [HACKERS] [patch] reorder tablespaces in basebackup tar stream for backup_label

2017-03-16 Thread Michael Paquier
On Fri, Mar 17, 2017 at 3:38 AM, Michael Banck wrote: >> Your patch would work with the stream mode though. > > Or, if not requesting the "WAL" option of the replication protocol's > BASE_BACKUP command. > > I agree it doesn't make sense to start messing with fetch mode, but I > don't think we gua

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2017-03-16 Thread Robert Haas
On Thu, Mar 16, 2017 at 9:31 PM, Michael Paquier wrote: > On Fri, Mar 17, 2017 at 12:19 AM, David Steele wrote: >> This patch does not apply cleanly at cccbdde: >> >> $ git apply ../other/0001-Rename-pg_clog-to-pg_xact.patch >> error: doc/src/sgml/ref/pg_resetxlog.sgml: No such file or directory

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2017-03-16 Thread Michael Paquier
On Fri, Mar 17, 2017 at 11:17 AM, Robert Haas wrote: > I understand that the point of renaming pg_clog to pg_xact is that > pg_clog contains the dreaded letters l-o-g, which we hypothesize > causes DBAs to remove it. (Alternate hypothesis: "So, that's what's > clogging my database!") > > Renaming

Re: [HACKERS] Potential data loss of 2PC files

2017-03-16 Thread Ashutosh Bapat
On Thu, Mar 16, 2017 at 10:17 PM, David Steele wrote: > On 2/13/17 12:10 AM, Michael Paquier wrote: >> On Tue, Jan 31, 2017 at 11:07 AM, Michael Paquier >> wrote: >>> On Mon, Jan 30, 2017 at 10:52 PM, Heikki Linnakangas >>> wrote: If that can happen, don't we have the same problem in many

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-16 Thread Craig Ringer
On 17 March 2017 at 08:10, Stas Kelvich wrote: > While working on this i’ve spotted quite a nasty corner case with aborted > prepared > transaction. I have some not that great ideas how to fix it, but maybe i > blurred my > view and missed something. So want to ask here at first. > > Suppose we

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-16 Thread Craig Ringer
On 16 March 2017 at 19:52, Stas Kelvich wrote: > > I’m working right now on issue with building snapshots for decoding prepared > tx. > I hope I'll send updated patch later today. Great. What approach are you taking? It looks like the snapshot builder actually does most of the work we need f

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-16 Thread Amit Kapila
On Thu, Mar 16, 2017 at 9:39 PM, Ashutosh Sharma wrote: >>> >> >> Don't you think, we should also clear it during the replay of >> XLOG_HASH_DELETE? We might want to log the clear of flag along with >> WAL record for XLOG_HASH_DELETE. >> > > Yes, it should be cleared. I completely missed this par

Re: [HACKERS] multivariate statistics (v25)

2017-03-16 Thread David Rowley
On 17 March 2017 at 11:20, Alvaro Herrera wrote: > (I think I lost some regression test files. I couldn't make up my mind > about putting each statistic type's tests in a separate file, or all > together in stats_ext.sql.) +1 for stats_ext.sql. I wanted to add some tests for pg_statisticsextdef(

[HACKERS] Re: [COMMITTERS] pgsql: Remove objname/objargs split for referring to objects

2017-03-16 Thread Peter Eisentraut
On 3/16/17 11:56, Alvaro Herrera wrote: > Michael Paquier wrote: > >> What are you using as CFLAGS? As both typenames should be normally >> set, what about initializing those fields with NULL and add an >> assertion like the attached? > > Actually, my compiler was right -- this was an ancient bug

Re: [HACKERS] possible encoding issues with libxml2 functions

2017-03-16 Thread Noah Misch
On Sun, Mar 12, 2017 at 10:26:33PM +0100, Pavel Stehule wrote: > 2017-03-12 21:57 GMT+01:00 Noah Misch : > > On Sun, Mar 12, 2017 at 08:36:58PM +0100, Pavel Stehule wrote: > > > 2017-03-12 0:56 GMT+01:00 Noah Misch : > > Please add a test case. > > It needs a application - currently there is not p

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-03-16 Thread Peter Eisentraut
On 3/16/17 14:55, Andres Freund wrote: > I indeed think that's the right consequence. One question is what to > replace it with exactly - are we guaranteed we can dynamically lookup > symbols by name in the main binary on every platform? I think there is probably a way to do this on all platforms

Re: [HACKERS] WAL Consistency checking for hash indexes

2017-03-16 Thread Amit Kapila
On Thu, Mar 16, 2017 at 1:15 PM, Ashutosh Sharma wrote: > Hi, > > Attached is the patch that allows WAL consistency tool to mask > 'LH_PAGE_HAS_DEAD_TUPLES' flag in hash index. The flag got added as a > part of 'Microvacuum support for Hash index' patch . I have already > tested it using Kuntal's

Re: [HACKERS] scram and \password

2017-03-16 Thread Michael Paquier
On Thu, Mar 16, 2017 at 10:52 PM, Heikki Linnakangas wrote: > On 03/14/2017 11:14 PM, Tom Lane wrote: >> >> In short, I don't think that argument refutes my position that "md5" >> in pg_hba.conf should be understood as allowing SCRAM passwords too. > > > Yeah, let's do that. Here's a patch. At le

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-16 Thread Andres Freund
On 2017-03-16 13:00:54 +0100, Petr Jelinek wrote: > On 15/03/17 17:55, Tom Lane wrote: > > Andrew Dunstan writes: > >> On 03/03/2017 11:11 PM, Tom Lane wrote: > >>> Yeah, I was wondering if this is just exposing a pre-existing bug. > >>> However, the "normal" path operates by repeatedly invoking P

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-16 Thread Tom Lane
Andres Freund writes: > On 2017-03-16 13:00:54 +0100, Petr Jelinek wrote: >> Looks like that didn't help either. >> >> I setup my own Windows machine and can reproduce the issue. I played >> around a bit and could not really find a fix other than adding >> WL_TIMEOUT and short timeout to WaitLatc

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-16 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of David Steele > The attached patch udpates the docs per your suggestion and has been rebased > on master at d69fae2. I made this ready for committer. The patch applied except for catversion.h, the

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-16 Thread Michael Paquier
On Fri, Mar 17, 2017 at 1:47 PM, Tsunakawa, Takayuki wrote: > BTW, does the developer of each feature have to modify the catalog version in > catversion.h? It's a bit annoying to see the patch application failure on > catversion.h. Committers take care of this part. > Isn't it enough to modif

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-16 Thread Tsunakawa, Takayuki
> From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Michael Paquier > On Fri, Mar 17, 2017 at 1:47 PM, Tsunakawa, Takayuki > wrote: > > BTW, does the developer of each feature have to modify the catalog version > in catversion.h? It's a bit annoyi

Re: [HACKERS] Speedup twophase transactions

2017-03-16 Thread Michael Paquier
On Thu, Mar 16, 2017 at 9:25 PM, Michael Paquier wrote: > On Thu, Mar 16, 2017 at 7:18 PM, Nikhil Sontakke > wrote: >>> + * * RecoverPreparedTransactions(), >>> StandbyRecoverPreparedTransactions() >>> + *and PrescanPreparedTransactions() have been modified to go >>> throug >>> + *

Re: [HACKERS] Radix tree for character conversion

2017-03-16 Thread Kyotaro HORIGUCHI
Thank you for committing this. At Mon, 13 Mar 2017 21:07:39 +0200, Heikki Linnakangas wrote in > On 03/13/2017 08:53 PM, Tom Lane wrote: > > Heikki Linnakangas writes: > >> It would be nice to run the map_checker tool one more time, though, to > >> verify that the mappings match those from Pos

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-03-16 Thread Kyotaro HORIGUCHI
At Tue, 7 Mar 2017 19:23:14 -0800, David Steele wrote in <3b7b7f90-db46-8c37-c4f7-443330c3a...@pgmasters.net> > On 3/3/17 4:54 PM, David Steele wrote: > > > On 2/1/17 1:25 AM, Kyotaro HORIGUCHI wrote: > >> Hello, thank you for moving this to the next CF. > >> > >> At Wed, 1 Feb 2017 13:09:51 +09

[HACKERS] <> join selectivity estimate question

2017-03-16 Thread Thomas Munro
Hi hackers, While studying a regression reported[1] against my parallel hash join patch, I noticed that we can also reach a good and a bad plan in unpatched master. One of the causes seems to be the estimated selectivity of a semi-join with an extra <> filter qual. Here are some times I measured

[HACKERS] Re: [BUGS] Problem in using pgbench's --connect(-C) and --rate=rate(-R rate) options together.

2017-03-16 Thread Fabien COELHO
Hello David, Repost from bugs. This patch does not apply at cccbdde: Indeed. It should not. The fix is for the 9.6 branch. The issue has been fixed by some heavy but very welcome restructuring in master. Marked as "Waiting for Author". I put it back to "Needs review". -- Fabien. --

Re: [HACKERS] increasing the default WAL segment size

2017-03-16 Thread Beena Emerson
Hello, Thank you for your comments, I will post an updated patch soon. On Fri, Mar 17, 2017 at 6:40 AM, Robert Haas wrote: > > > +assign_wal_segment_size(int newval, void *extra) > > Why does a PGC_INTERNAL GUC need an assign hook? I think the GUC > should only be there to expose the value; it

Re: [HACKERS] [PATCH]: fix bug in SP-GiST box_ops

2017-03-16 Thread Kyotaro HORIGUCHI
At Fri, 10 Mar 2017 12:15:52 +0300, Nikita Glukhov wrote in <48f6934b-b994-4aa2-b6ad-aaa4f5a12...@postgrespro.ru> > On 10.03.2017 02:13, Tels wrote: > > > I can't comment on the code, but the grammar on the comments caught my > > eye: > >> +/* Can any range from range_box does not extend higher

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-16 Thread Fabien COELHO
Hello Corey & Tom, What is not done: - skipped slash commands still consume the rest of the line That last part is big, to quote Tom: * More generally, I do not think that the approach of having exec_command simply fall out immediately when in a false branch is going to work, because it ignor

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-03-16 Thread Amit Kapila
On Sun, Mar 12, 2017 at 8:11 AM, Robert Haas wrote: > On Fri, Mar 10, 2017 at 7:39 PM, Amit Kapila wrote: >> I agree that more analysis can help us to decide if we can use subxids >> from PGPROC and if so under what conditions. Have you considered the >> another patch I have posted to fix the is

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-16 Thread Ashutosh Sharma
On Fri, Mar 17, 2017 at 8:20 AM, Amit Kapila wrote: > On Thu, Mar 16, 2017 at 9:39 PM, Ashutosh Sharma > wrote: >>> >>> Don't you think, we should also clear it during the replay of >>> XLOG_HASH_DELETE? We might want to log the clear of flag along with >>> WAL record for XLOG_HASH_DELETE.

<    1   2