Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-03-31 Thread Magnus Hagander
On Wed, Mar 29, 2017 at 1:05 PM, Michael Banck wrote: > Hi, > > Am Montag, den 27.02.2017, 16:20 +0100 schrieb Magnus Hagander: > > On Sun, Feb 26, 2017 at 9:59 PM, Tom Lane wrote: > > Is there an argument for back-patching this? > > > > > > Seems you were typing that at the same time as

Re: [HACKERS] Typo in libpq

2017-03-31 Thread Magnus Hagander
On Thu, Mar 30, 2017 at 3:48 PM, Daniel Gustafsson wrote: > There seems to be a typo in libpq as per attached, “..we will loose error > messages” should probably be “..we will lose error messages”. > Applied, thanks. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-l

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Dilip Kumar
On Thu, Mar 30, 2017 at 5:27 PM, Amit Kapila wrote: > I am not sure if we can consider it as completely synthetic because we > might see some similar cases for json datatypes. Can we once try to > see the impact when the same test runs from multiple clients? For > your information, I am also try

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-03-31 Thread Michael Paquier
On Wed, Mar 8, 2017 at 10:39 PM, Robert Haas wrote: > On Tue, Mar 7, 2017 at 10:01 PM, Michael Paquier > wrote: >> This way, we can be sure that two UTf-8 strings are considered as >> equivalent in a SASL exchange, in our case we care about the password >> string (we should care about the usernam

[HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-03-31 Thread Tsunakawa, Takayuki
Hello, I found a trivial bug that terminates the connection. The attached patch fixes this. PROBLEM Savepoint-related statements in a multi-command query terminates the connection unexpectedly, as follows. $ psql -d postgres -c "SELECT 1; SAVEPOINT s

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-31 Thread Amit Langote
Horiguchi-san, On 2017/03/31 15:50, Kyotaro HORIGUCHI wrote: > At Thu, 30 Mar 2017 20:58:35 +0900, Amit Langote wrote: >> Updated patch attached. > > Thank you. > > - Applies cleanly on master (f90d23d) > - Compiled without error > - Code seems fine. > - Documentaion seems fine.. for me. > - Pas

[HACKERS] Parallel query execution with SPI

2017-03-31 Thread Konstantin Knizhnik
Hi hackers, It is possible to execute query concurrently using SPI? If so, how it can be enforced? I tried to open cursor with CURSOR_OPT_PARALLEL_OK flag but it doesn't help: query is executed by single backend while the same query been launched at top level uses parallel plan: fsstate->

Re: [HACKERS] Allow interrupts on waiting standby

2017-03-31 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:michael.paqu...@gmail.com] > Oops, sorry for that, I quite mess up with this patch. The WaitLatch() call > should still have WL_POSTMASTER_DEATH so as it can leave earlier, but yes > I agree with your analysis that HandleStartupProcInterrupts() as this is > part of the

[HACKERS] Foreign tables don't enforce the partition constraint

2017-03-31 Thread Amit Langote
We don't enforce the constraints defined on foreign tables in ExecInsert() and ExecUpdate(). (COPY FROM does not support foreign tables at all.) Since partition constraints are enforced using ExecConstraints() which is not called for foreign tables, they will not be checked if one inserts directly

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2017-03-31 Thread Andreas Karlsson
Thanks for the feedback. I will look at it when I get the time. On 03/31/2017 08:27 AM, Michael Paquier wrote: - Do a per-index rebuild and not a per-relation rebuild for concurrent indexing. Doing a per-relation reindex has the disadvantage that many objects need to be created at the same time,

Re: [HACKERS] multivariate statistics (v25)

2017-03-31 Thread Kyotaro HORIGUCHI
Hello, At Fri, 31 Mar 2017 03:03:06 +1300, David Rowley wrote in > On 25 March 2017 at 07:35, Alvaro Herrera wrote: > > > As I said in another thread, I pushed parts 0002,0003,0004. Tomas said > > he would try to rebase patches 0001,0005,0006 on top of what was > > committed. My intention i

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-31 Thread Kyotaro HORIGUCHI
At Fri, 31 Mar 2017 16:17:05 +0900, Amit Langote wrote in <2dec1acb-6e2f-5aa5-0e26-fcc172ce9...@lab.ntt.co.jp> > Horiguchi-san, > > On 2017/03/31 15:50, Kyotaro HORIGUCHI wrote: > > At Thu, 30 Mar 2017 20:58:35 +0900, Amit Langote wrote: > >> Updated patch attached. > > > > Thank you. > > > >

Re: [HACKERS] UPDATE of partition key

2017-03-31 Thread Amit Langote
Hi Amit, Thanks for the updated patches. On 2017/03/28 19:12, Amit Khandekar wrote: > On 27 March 2017 at 13:05, Amit Khandekar wrote: >>> Also, there are a few places in the documentation mentioning that such >>> updates cause error, >>> which will need to be updated. Perhaps also add some ex

Re: [HACKERS] Something broken around FDW connection close

2017-03-31 Thread David Rowley
On 31 March 2017 at 16:32, Etsuro Fujita wrote: > On 2017/03/31 8:28, David Rowley wrote: > >> create table t (a int, b int); >> insert into t1 select x/100,x/100 from generate_series(1,10) x; >> create extension if not exists postgres_fdw; >> create server test_server foreign data wrapper p

Re: [HACKERS] multivariate statistics (v25)

2017-03-31 Thread David Rowley
On 31 March 2017 at 21:18, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, > > At Fri, 31 Mar 2017 03:03:06 +1300, David Rowley < > david.row...@2ndquadrant.com> wrote in T5JLce5ynCi1vvezXxX=w...@mail.gmail.com> > > FWIW, I tries this. This cleanly applied on it but make ends

Re: [HACKERS] Foreign tables don't enforce the partition constraint

2017-03-31 Thread Ashutosh Bapat
On Fri, Mar 31, 2017 at 1:36 PM, Amit Langote wrote: > We don't enforce the constraints defined on foreign tables in ExecInsert() > and ExecUpdate(). (COPY FROM does not support foreign tables at all.) > Since partition constraints are enforced using ExecConstraints() which is > not called for fo

Re: [HACKERS] show "aggressive" or not in autovacuum logs

2017-03-31 Thread Masahiko Sawada
On Wed, Mar 29, 2017 at 12:46 PM, Kyotaro HORIGUCHI wrote: > Hello, it would be too late but I'd like to propose this because > this cannot be back-patched. > > > In autovacuum logs, "%u skipped frozen" shows the number of pages > skipped by ALL_FROZEN only in aggressive vacuum. > > So users canno

Re: [HACKERS] ANALYZE command progress checker

2017-03-31 Thread Masahiko Sawada
On Thu, Mar 30, 2017 at 6:19 PM, vinayak wrote: > > On 2017/03/30 17:39, Masahiko Sawada wrote: >> >> On Wed, Mar 29, 2017 at 5:38 PM, vinayak >> wrote: >>> >>> On 2017/03/25 4:30, Robert Haas wrote: On Fri, Mar 24, 2017 at 3:41 AM, vinayak wrote: > > I have updated the pa

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-03-31 Thread Alexander Korotkov
On Sun, Mar 26, 2017 at 12:29 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Sat, Mar 25, 2017 at 11:32 PM, Tom Lane wrote: > >> Alexander Korotkov writes: >> > I moved PPC implementation of pg_atomic_fetch_mask_add_u32() into >> > port/atomics/arch-ppc.h. I also had to declare

Re: [HACKERS] Parallel query execution with SPI

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 3:33 AM, Konstantin Knizhnik wrote: > It is possible to execute query concurrently using SPI? > If so, how it can be enforced? > I tried to open cursor with CURSOR_OPT_PARALLEL_OK flag but it doesn't help: > query is executed by single backend while the same query been laun

Re: [HACKERS] WIP: Covering + unique indexes.

2017-03-31 Thread Robert Haas
On Thu, Mar 30, 2017 at 5:22 PM, Anastasia Lubennikova wrote: > Well, > I don't know how can we estimate the quality of the review or testing. > The patch was reviewed by many people. > Here are those who marked themselves as reviewers on this and previous > committfests: Stephen Frost (sfrost), A

Re: [HACKERS] Supporting huge pages on Windows

2017-03-31 Thread Amit Kapila
On Fri, Mar 31, 2017 at 8:05 AM, Tsunakawa, Takayuki wrote: > From: Amit Kapila [mailto:amit.kapil...@gmail.com] >> The latest patch looks good to me apart from one Debug message, so I have >> marked it as Ready For Committer. > > Thank you so much! > > >> + ereport(DEBUG1, >> + (errmsg("disabling

Re: [HACKERS] Parallel query execution with SPI

2017-03-31 Thread Rafia Sabih
On Fri, Mar 31, 2017 at 4:18 PM, Robert Haas wrote: > On Fri, Mar 31, 2017 at 3:33 AM, Konstantin Knizhnik > wrote: >> It is possible to execute query concurrently using SPI? >> If so, how it can be enforced? >> I tried to open cursor with CURSOR_OPT_PARALLEL_OK flag but it doesn't help: >> query

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-03-31 Thread Amit Kapila
On Thu, Mar 30, 2017 at 8:24 AM, Robert Haas wrote: > On Wed, Mar 29, 2017 at 8:00 PM, Tomas Vondra > wrote: >> What is however strange is that changing max_parallel_workers_per_gather >> affects row estimates *above* the Gather node. That seems a bit, um, >> suspicious, no? See the parallel-esti

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-03-31 Thread Corey Huinker
On Thu, Mar 30, 2017 at 1:33 PM, Tom Lane wrote: > single-quoted according to Unix shell conventions. (So the > processing would be a bit different from what it is for the > same notation in SQL contexts.) > +1 Having been bit by format '%L' prepending an 'E' to any string that happens to have

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Simon Riggs
On 30 March 2017 at 16:50, Robert Haas wrote: > On Thu, Mar 30, 2017 at 11:41 AM, Andres Freund wrote: >> On 2017-03-30 16:43:41 +0530, Pavan Deolasee wrote: >>> Looks like OID conflict to me.. Please try rebased set. >> >> Pavan, Alvaro, everyone: I know you guys are working very hard on this, >

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-03-31 Thread Rafia Sabih
On Fri, Mar 31, 2017 at 5:13 PM, Amit Kapila wrote: > On Thu, Mar 30, 2017 at 8:24 AM, Robert Haas wrote: >> On Wed, Mar 29, 2017 at 8:00 PM, Tomas Vondra >> wrote: >>> What is however strange is that changing max_parallel_workers_per_gather >>> affects row estimates *above* the Gather node. Tha

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-03-31 Thread Ashutosh Bapat
Please add this to the next commitfest. I think there's some misunderstanding between exec_simple_query() and the way we manage transaction block state machine. In exec_simple_query() 952 * We'll tell PortalRun it's a top-level command iff there's exactly one 953 * raw parsetree. If

Re: [HACKERS] postgres_fdw bug in 9.6

2017-03-31 Thread Etsuro Fujita
On 2017/03/30 20:16, Ashutosh Bapat wrote: The patch applies cleanly, compiles. make check in regress as well as postgres_fdw works fine. Here are few comments Thanks for the review! local-join should be local join. OK, done. The comments should explain why. +/* Should be unparame

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-03-31 Thread Daniel Verite
Tom Lane wrote: > Thoughts? ISTM that expr is too painful to use to be seen as the idiomatic way of achieving comparison in psql. Among its disadvantages, it won't work on windows, and its interface is hard to work with due to the necessary quoting of half its operators, and the mandator

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 7:53 AM, Simon Riggs wrote: > So Andres says defer this, but Robert says "more review", which is > more than just deferral. > > We have some risky things in this release such as Hash Indexes, > function changes. I perfectly understand that perception of risk is > affected s

Re: [HACKERS] Parallel query execution with SPI

2017-03-31 Thread Konstantin Knizhnik
On 31.03.2017 13:48, Robert Haas wrote: On Fri, Mar 31, 2017 at 3:33 AM, Konstantin Knizhnik wrote: It is possible to execute query concurrently using SPI? If so, how it can be enforced? I tried to open cursor with CURSOR_OPT_PARALLEL_OK flag but it doesn't help: query is executed by single b

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-03-31 Thread Robert Haas
On Thu, Mar 30, 2017 at 4:35 PM, Kuntal Ghosh wrote: > 2. the server restarts automatically, initialize > BackgroundWorkerData->parallel_register_count and > BackgroundWorkerData->parallel_terminate_count in the shared memory. > After that, it calls ForgetBackgroundWorker and it increments > paral

Re: [HACKERS] REFERENCES privilege should not be symmetric (was Re: [GENERAL] Postgres Permissions Article)

2017-03-31 Thread Robert Haas
On Thu, Mar 30, 2017 at 4:45 PM, Tom Lane wrote: > In short, it seems like this statement in the docs is correctly describing > our code's behavior, but said behavior is wrong and should be changed. > I'd propose fixing it like that in HEAD; I'm not sure if the back branches > should also be chang

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-03-31 Thread Robert Haas
On Tue, Mar 28, 2017 at 7:39 PM, Petr Jelinek wrote: > Sigh, forgot git add for the docs, so one more try... +if (strncmp(worker->bgw_library_name, "postgres", 8) != 0) +return NULL; I think that's not right. You don't want to match postgresshdkgjsdglkjs. Aside from that, these loo

Re: [HACKERS] Multiple false-positive warnings from Valgrind

2017-03-31 Thread Aleksander Alekseev
Hi Kyotaro, > > And it seems to me that this is caused by the routines of OpenSSL. > > When building without --with-openssl, using the fallback > > implementations of SHA256 and RAND_bytes I see no warnings generated > > by scram_build_verifier... I think it makes most sense to discard that > > fr

Re: [HACKERS] Fix obsolete comment in GetSnapshotData

2017-03-31 Thread Robert Haas
On Wed, Mar 29, 2017 at 12:00 AM, Craig Ringer wrote: > There's an outdated reference to GetOldestXmin(true, true) in > GetSnapshotData. It hasn't had that call signature for a long while > now. Update the comment to reflect the current signature. > > diff --git a/src/backend/storage/ipc/procarray

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-03-31 Thread Jan Michálek
2017-03-31 12:01 GMT+02:00 Pierre Ducroquet : > On Friday, March 31, 2017 7:17:08 AM CEST, Jan Michálek wrote: > >> >> >> 2017-03-30 21:53 GMT+02:00 Pavel Stehule : >> >> >> 2017-03-29 20:11 GMT+02:00 Jan Michálek : >> >> >> 2017-03-27 19:41 GMT+02:00 Jan Michálek : >> >> >> 2017-03-23 17:26 GMT+0

[HACKERS] [PATCH] Remove unused argument in btree_xlog_split

2017-03-31 Thread Aleksander Alekseev
Hi, Turned out that there is an unused argument `isroot` in `btree_xlog_split` procedure. Suggested patch fixes it. This issue was discovered by Anastasia Lubennikova, coding is done by me. -- Best regards, Aleksander Alekseev diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/acces

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-31 Thread David Steele
On 3/29/17 2:23 AM, Masahiko Sawada wrote: On Wed, Mar 29, 2017 at 12:23 AM, David Steele wrote: On 3/23/17 1:54 AM, Masahiko Sawada wrote: On Wed, Mar 15, 2017 at 7:51 AM, Peter Geoghegan wrote: On Tue, Mar 14, 2017 at 3:10 PM, Peter Geoghegan wrote: We already have BTPageOpaqueData.bt

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Pavan Deolasee
On Fri, Mar 31, 2017 at 6:47 PM, Robert Haas wrote: > > > 2. WARM is a non-optional feature which touches the on-disk format. > There is nothing more dangerous than that. If hash indexes have bugs, > people can avoid those bugs by not using them; there are good reasons > to suppose that hash inde

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-31 Thread David Steele
On 3/25/17 12:12 AM, Peter Eisentraut wrote: I'm wondering if this is a perl version/platform issue around $tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/; where we're not recognising the required output from psql when we get it. What's in src/test/recovery/tmp_check/log/regress_log_011*

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-03-31 Thread Pavel Stehule
2017-03-31 15:00 GMT+02:00 Daniel Verite : > Tom Lane wrote: > > > Thoughts? > > ISTM that expr is too painful to use to be seen as the > idiomatic way of achieving comparison in psql. > > Among its disadvantages, it won't work on windows, and its > interface is hard to work with due to th

Re: [HACKERS] BRIN cost estimate

2017-03-31 Thread David Steele
On 3/26/17 7:44 AM, Emre Hasegeli wrote: If we want to have a variable which stores the number of ranges, then I think numRanges is better than numBlocks. I can't imagine many people would disagree there. I renamed it with other two. At the very least please write a comment to explain this in

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-31 Thread Robert Haas
On Wed, Mar 29, 2017 at 2:23 AM, Masahiko Sawada wrote: > I was thinking that the status of this patch is still "Needs review" > because I sent latest version patch[1]. I think you're right. I took a look at this today. I think there is some problem with the design of this patch. I originally

Re: [HACKERS] Adding support for Default partition in partitioning

2017-03-31 Thread David Steele
On 3/29/17 8:13 AM, Rahila Syed wrote: Thanks for reporting. I have identified the problem and have a fix. Currently working on allowing adding a partition after default partition if the default partition does not have any conflicting rows. Will update the patch with both of these. The CF has

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-31 Thread Craig Ringer
On 31 Mar. 2017 22:31, "David Steele" wrote: On 3/25/17 12:12 AM, Peter Eisentraut wrote: > I'm wondering if this is a perl version/platform issue around >> >> $tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/; >> >> where we're not recognising the required output from psql when we get it. >>

[HACKERS] Re: PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-31 Thread David Steele
On 3/29/17 11:08 AM, Tomas Vondra wrote: The attached patch is the best I came up with - it essentially shares just the tuple-forming part, which is exactly the same in both cases. I have marked this submission "Needs Review". -- -David da...@pgmasters.net -- Sent via pgsql-hackers mailing

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-03-31 Thread Pierre Ducroquet
On Friday, March 31, 2017 7:17:08 AM CEST, Jan Michálek wrote: 2017-03-30 21:53 GMT+02:00 Pavel Stehule : 2017-03-29 20:11 GMT+02:00 Jan Michálek : 2017-03-27 19:41 GMT+02:00 Jan Michálek : 2017-03-23 17:26 GMT+01:00 Pierre Ducroquet : The following review has been posted through the comm

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-31 Thread David Steele
Hi, On 3/30/17 2:12 PM, Daniel Verite wrote: Vaishnavi Prabakaran wrote: Hmm, With batch mode, after sending COPY command to server(and server started processing the query and goes into COPY state) , client does not immediately read the result , but it keeps sending other queries to th

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-31 Thread David Steele
On 3/31/17 10:46 AM, Craig Ringer wrote: Patches 1 and 2 were the key parts and thanks to Robert's helpful review, advice and edits they're committed now. Committed, done. Yay. Excellent. I have marked this a "Committed" by Robert. One down... -- -David da...@pgmasters.net -- Sent via pg

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-03-31 Thread Tom Lane
Corey Huinker writes: > On Thu, Mar 30, 2017 at 1:33 PM, Tom Lane wrote: >> single-quoted according to Unix shell conventions. (So the >> processing would be a bit different from what it is for the >> same notation in SQL contexts.) > Any reason we wouldn't do :"VARIABLE" as well? Well, what w

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-03-31 Thread Tom Lane
"Daniel Verite" writes: > ISTM that expr is too painful to use to be seen as the > idiomatic way of achieving comparison in psql. I'm not proposing it as the best permanent solution, just saying that having this in v10 is a lot better than having nothing in v10. And we certainly aren't going to g

Re: [HACKERS] multivariate statistics (v25)

2017-03-31 Thread David Rowley
On 31 March 2017 at 21:18, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > > When adding these two parameters I had 2nd thoughts that the > "tryextstats" > > was required at all. We could just have this controlled by if the rel is > a > > base rel of kind RTE_RELATION. I ended up hav

Re: [HACKERS] REFERENCES privilege should not be symmetric (was Re: [GENERAL] Postgres Permissions Article)

2017-03-31 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 30, 2017 at 4:45 PM, Tom Lane wrote: >> In short, it seems like this statement in the docs is correctly describing >> our code's behavior, but said behavior is wrong and should be changed. >> I'd propose fixing it like that in HEAD; I'm not sure if the back branc

[HACKERS] pg_partman 3.0.0 - real-world usage of native partitioning and a case for native default

2017-03-31 Thread Keith Fiske
I've gotten pg_partman working with native partitioning already so I can hopefully have things ready to work when 10 is released. I've got a branch on github with this version for anyone to test and I'll hopefully have this released in the next few weeks after I finish some more testing myself. Any

Re: [HACKERS] [patch] reorder tablespaces in basebackup tar stream for backup_label

2017-03-31 Thread Michael Banck
On Fri, Mar 31, 2017 at 02:11:44PM +0900, Kyotaro HORIGUCHI wrote: > At Fri, 31 Mar 2017 13:37:38 +0900, Michael Paquier > wrote in > > > In my first reviews of the patch, I completely forgot the fact that > > BASE_BACKUP does send the start LSN of the backup in the first result > > set, so the

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

2017-03-31 Thread David Steele
Hi Arthur. On 3/23/17 8:45 AM, Etsuro Fujita wrote: On 2017/03/21 18:38, Etsuro Fujita wrote: On 2017/03/16 22:23, Arthur Zakirov wrote: Can you rebase the patch? It is not applied now. Ok, will do. Thanks for the report! Done. Also, I added regression tests and revised code and comment

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

2017-03-31 Thread Arthur Zakirov
Hello, On 31.03.2017 18:47, David Steele wrote: Hi Arthur. On 3/23/17 8:45 AM, Etsuro Fujita wrote: Done. Also, I added regression tests and revised code and comments a bit. (As for create_foreignscan_path(), I haven't done anything about that yet.) Please find attached a new version creat

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-31 Thread Ashutosh Sharma
Hi, On Wed, Mar 29, 2017 at 8:38 PM, Tomas Vondra wrote: > > > On 03/24/2017 04:27 AM, Peter Eisentraut wrote: >> >> On 3/17/17 18:35, Tomas Vondra wrote: >>> >>> On 03/17/2017 05:23 PM, Peter Eisentraut wrote: I'm struggling to find a good way to share code between bt_page_items(t

Re: [HACKERS] delta relations in AFTER triggers

2017-03-31 Thread Kevin Grittner
On Thu, Mar 23, 2017 at 7:14 PM, Thomas Munro wrote: > my only other comment would be a bikeshed issue: > Enr isn't a great name for a struct. I know, but EphemeralNamedRelation starts to get kinda long, especially when making the normal sorts of concatenations. I started making the change and

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-03-31 Thread Alvaro Herrera
Ashutosh Bapat wrote: > Please add this to the next commitfest. If this cannot be reproduced in 9.6, then it must be added to the Open Items wiki page instead. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services --

Re: [HACKERS] postgres_fdw bug in 9.6

2017-03-31 Thread Jeff Janes
On Thu, Mar 23, 2017 at 5:20 AM, Etsuro Fujita wrote: > On 2017/03/21 18:40, Etsuro Fujita wrote: > >> Ok, I'll update the patch. One thing I'd like to revise in addition to >> that is (1) add to JoinPathExtraData a flag member to indicate whether >> to give the FDW a chance to consider a remote

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-03-31 Thread Tom Lane
Corey Huinker writes: > On Thu, Mar 30, 2017 at 1:33 PM, Tom Lane wrote: >> single-quoted according to Unix shell conventions. (So the >> processing would be a bit different from what it is for the >> same notation in SQL contexts.) > +1 Here's a proposed patch for this. I used the existing a

Re: [HACKERS] delta relations in AFTER triggers

2017-03-31 Thread Kevin Grittner
On Thu, Mar 30, 2017 at 11:51 AM, Kevin Grittner wrote: > New version attached. It needs some of these problem cases added to > the testing, and a mention in the docs that only C and plpgsql > triggers can use the feature so far. I'll add those tomorrow. Done and attached. Now the question is

[HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Andres Freund
Hi, The covering indexes patch [1] really needs a version of heap_form_tuple/index_form_tuple that allows to specify the number of columns in the to-be generated tuple. Previously the faster expression evaluation stuff could also have benefited form the same for both forming and deforming tuples.

Re: [HACKERS] REFERENCES privilege should not be symmetric (was Re: [GENERAL] Postgres Permissions Article)

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 11:29 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Mar 30, 2017 at 4:45 PM, Tom Lane wrote: >>> In short, it seems like this statement in the docs is correctly describing >>> our code's behavior, but said behavior is wrong and should be changed. >>> I'd propose fi

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 11:08 AM, David Steele wrote: > On 3/31/17 10:46 AM, Craig Ringer wrote: >> Patches 1 and 2 were the key parts and thanks to Robert's helpful >> review, advice and edits they're committed now. >> >> Committed, done. Yay. > > Excellent. I have marked this a "Committed" by R

Re: [HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 1:24 PM, Andres Freund wrote: > The covering indexes patch [1] really needs a version of > heap_form_tuple/index_form_tuple that allows to specify the number of > columns in the to-be generated tuple. Previously the faster expression > evaluation stuff could also have bene

Re: [HACKERS] Somebody has not thought through subscription locking considerations

2017-03-31 Thread Tom Lane
Masahiko Sawada writes: > On Fri, Mar 31, 2017 at 9:53 AM, Petr Jelinek > wrote: >> On 30/03/17 07:25, Tom Lane wrote: >>> I await with interest an explanation of what "VACUUM FULL pg_class" is >>> doing trying to acquire ShareRowExclusiveLock on pg_subscription_rel, not >>> to mention why a DROP

[HACKERS] Some never executed code regarding the table sync worker

2017-03-31 Thread Masahiko Sawada
Hi all, After launched the sync table worker it enters ApplyWorkerMain function. And then the table sync worker calls LogicalRepSyncTableStart to synchronize the target table. In LogicalRepSyncTableStart, finish_sync_worker is always called and then the table sync worker process always exits after

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-31 Thread Tomas Vondra
On 03/31/2017 06:01 PM, Ashutosh Sharma wrote: It seems like the latest patch(v4) shared by Tomas upthread is an empty patch. If I am not wrong, please share the correct patch. Thanks. OMG, this is the second time I managed to generate an empty patch. I really need to learn not to do that ..

Re: [HACKERS] REFERENCES privilege should not be symmetric (was Re: [GENERAL] Postgres Permissions Article)

2017-03-31 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 31, 2017 at 11:29 AM, Tom Lane wrote: >> The argument for not back-patching a bug fix usually boils down to >> fear of breaking existing applications, but it's hard to see how >> removal of a permission check could break a working application --- >> especially wh

Re: [HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Tom Lane
Andres Freund writes: > The covering indexes patch [1] really needs a version of > heap_form_tuple/index_form_tuple that allows to specify the number of > columns in the to-be generated tuple. I was thinking about that this morning, and wondering why exactly it would need such a thing. Certainly

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Robert Haas
On Thu, Mar 30, 2017 at 10:49 AM, Petr Jelinek wrote: > While reading this thread I am thinking if we could just not do WARM on > TOAST and compressed values if we know there might be regressions there. > I mean I've seen the problem WARM tries to solve mostly on timestamp or > boolean values and

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 10:24 AM, Pavan Deolasee wrote: > Having worked on it for some time now, I can say that WARM uses pretty much > the same infrastructure that HOT uses for cleanup/pruning tuples from the > heap. So the risk of having a bug which can eat your data from the heap is > very low.

Re: [HACKERS] WIP: Covering + unique indexes.

2017-03-31 Thread Andres Freund
On 2017-03-31 20:40:59 +0300, Anastasia Lubennikova wrote: > 30.03.2017 22:11, Andres Freund > > Any objection from reviewers to push both patches? > > First of all, I want to thank you and Robert for reviewing this patch. > Your expertise in postgres subsystems is really necessary for features li

Re: [HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 1:44 PM, Tom Lane wrote: > Andres Freund writes: >> The covering indexes patch [1] really needs a version of >> heap_form_tuple/index_form_tuple that allows to specify the number of >> columns in the to-be generated tuple. > > I was thinking about that this morning, and wo

Re: [HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Andres Freund
On 2017-03-31 13:44:52 -0400, Tom Lane wrote: > Andres Freund writes: > > The covering indexes patch [1] really needs a version of > > heap_form_tuple/index_form_tuple that allows to specify the number of > > columns in the to-be generated tuple. > > I was thinking about that this morning, and wo

Re: [HACKERS] delta relations in AFTER triggers

2017-03-31 Thread Tom Lane
Kevin Grittner writes: > On Thu, Mar 23, 2017 at 7:14 PM, Thomas Munro > wrote: >> my only other comment would be a bikeshed issue: >> Enr isn't a great name for a struct. > I know, but EphemeralNamedRelation starts to get kinda long, > especially when making the normal sorts of concatenations.

Re: [HACKERS] REFERENCES privilege should not be symmetric (was Re: [GENERAL] Postgres Permissions Article)

2017-03-31 Thread David G. Johnston
On Fri, Mar 31, 2017 at 10:40 AM, Tom Lane wrote: > Robert Haas writes: > > On Fri, Mar 31, 2017 at 11:29 AM, Tom Lane wrote: > >> The argument for not back-patching a bug fix usually boils down to > >> fear of breaking existing applications, but it's hard to see how > >> removal of a permissio

Re: [HACKERS] WIP: Covering + unique indexes.

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 1:40 PM, Anastasia Lubennikova wrote: > First of all, I want to thank you and Robert for reviewing this patch. > Your expertise in postgres subsystems is really necessary for features like > this. > I just wonder, why don't you share your thoughts and doubts till the "last

Re: [HACKERS] REFERENCES privilege should not be symmetric (was Re: [GENERAL] Postgres Permissions Article)

2017-03-31 Thread Magnus Hagander
On Fri, Mar 31, 2017 at 7:40 PM, Tom Lane wrote: > Robert Haas writes: > > On Fri, Mar 31, 2017 at 11:29 AM, Tom Lane wrote: > >> The argument for not back-patching a bug fix usually boils down to > >> fear of breaking existing applications, but it's hard to see how > >> removal of a permission

Re: [HACKERS] delta relations in AFTER triggers

2017-03-31 Thread Robert Haas
On Fri, Mar 31, 2017 at 1:50 PM, Tom Lane wrote: > Kevin Grittner writes: >> On Thu, Mar 23, 2017 at 7:14 PM, Thomas Munro >> wrote: >>> my only other comment would be a bikeshed issue: >>> Enr isn't a great name for a struct. > >> I know, but EphemeralNamedRelation starts to get kinda long, >>

Re: [HACKERS] WIP: Covering + unique indexes.

2017-03-31 Thread Anastasia Lubennikova
31.03.2017 20:47, Andres Freund: Maybe it would be better to modify index_form_tuple() to accept a new argument with a number of attributes, then you can just Assert that this number is never higher than the number of attributes in the TupleDesc. Good point. I agree that this function is a bit s

[HACKERS] merge psql ef/ev sf/sv handling functions

2017-03-31 Thread Fabien COELHO
Hello, While reviewing Corey's \if patch, I complained that there was some amount of copy-paste in "psql/command.c". Here is an attempt at merging some functions which removes 160 lines of code. -- Fabien.diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c index 94a3cfc..edf875d 1

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-31 Thread Masahiko Sawada
On Fri, Mar 31, 2017 at 11:44 PM, Robert Haas wrote: > On Wed, Mar 29, 2017 at 2:23 AM, Masahiko Sawada > wrote: >> I was thinking that the status of this patch is still "Needs review" >> because I sent latest version patch[1]. > > I think you're right. > > I took a look at this today. I think

Re: [HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Tom Lane
Andres Freund writes: > On 2017-03-31 13:44:52 -0400, Tom Lane wrote: >> Andres Freund writes: >>> The covering indexes patch [1] really needs a version of >>> heap_form_tuple/index_form_tuple that allows to specify the number of >>> columns in the to-be generated tuple. >> I was thinking about

Re: [HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Tom Lane
Robert Haas writes: > You might want to have a read through that patch. I think your > opinion would be helpful in determining how close that patch is to > being ready to commit (same for WARM). Well, now that we have an extra week, maybe I'll find the time. regards, tom

Re: [HACKERS] Somebody has not thought through subscription locking considerations

2017-03-31 Thread Petr Jelinek
On 31/03/17 19:35, Tom Lane wrote: > Masahiko Sawada writes: >> On Fri, Mar 31, 2017 at 9:53 AM, Petr Jelinek >> wrote: >>> On 30/03/17 07:25, Tom Lane wrote: I await with interest an explanation of what "VACUUM FULL pg_class" is doing trying to acquire ShareRowExclusiveLock on pg_subsc

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-03-31 Thread Mike Palmiotto
On Mon, Mar 27, 2017 at 12:09 PM, Mike Palmiotto wrote: > On Mon, Mar 27, 2017 at 11:46 AM, Robert Haas wrote: >> >> Note that sepgsql hasn't been updated to work with RLS yet, either, >> but we didn't regard that as an open item for RLS, or if we did the >> resolution was just to document it.

[HACKERS] bumping HASH_VERSION to 3

2017-03-31 Thread Robert Haas
Starting a new thread about this to get more visibility. Despite the extensive work that has been done on hash indexes this release, we have thus far not made any change to the on-disk format that is not nominally backward-compatible. Commit 293e24e507838733aba4748b514536af2d39d7f2 did make a cha

Re: [HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Peter Geoghegan
On Fri, Mar 31, 2017 at 2:11 PM, Tom Lane wrote: >> The patch does actually store truncated/key-only tuples in the hi keys / >> non-leaf-nodes, which don't need the "included" columns. > > Hm. Since index tuples lack any means of indicating the actual number > of columns included (ie there's no e

Re: [HACKERS] delta relations in AFTER triggers

2017-03-31 Thread Kevin Grittner
On Fri, Mar 31, 2017 at 12:58 PM, Robert Haas wrote: > On Fri, Mar 31, 2017 at 1:50 PM, Tom Lane wrote: >> I would vote for calling the struct typedef EphemeralNamedRelation and >> using the abbreviation ENR (capitalized that way, not as Enr) in related >> function names, where that seemed sensi

Re: [HACKERS] bumping HASH_VERSION to 3

2017-03-31 Thread Magnus Hagander
On Fri, Mar 31, 2017 at 8:17 PM, Robert Haas wrote: > Starting a new thread about this to get more visibility. > > Despite the extensive work that has been done on hash indexes this > release, we have thus far not made any change to the on-disk format > that is not nominally backward-compatible.

[HACKERS] [BUGS] BUG #14600: Passwords in user mappings leaked by psql \deu+ command

2017-03-31 Thread Feike Steenbergen
Forwarding message from pgsql-bugs for review Attached a patch which copies the logic from commit 93a6be63a55a8cd0d73b3fa81eb6a46013a3a974. In the current implementation we only consider privileges of the foreign server in determining whether or not to show the user mapping details. This patch c

Re: [HACKERS] Somebody has not thought through subscription locking considerations

2017-03-31 Thread Tom Lane
Petr Jelinek writes: > On 31/03/17 19:35, Tom Lane wrote: >> At the very least, it would be a good idea to exclude the system >> catalogs from logical replication, wouldn't it? > They are excluded. Well, the exclusion is apparently not checked early enough. I would say that touches of system ca

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Pavan Deolasee
On Fri, Mar 31, 2017 at 11:16 PM, Robert Haas wrote: > On Fri, Mar 31, 2017 at 10:24 AM, Pavan Deolasee > wrote: > > Having worked on it for some time now, I can say that WARM uses pretty > much > > the same infrastructure that HOT uses for cleanup/pruning tuples from the > > heap. So the risk o

Re: [HACKERS] bumping HASH_VERSION to 3

2017-03-31 Thread Jesper Pedersen
On 03/31/2017 02:17 PM, Robert Haas wrote: Starting a new thread about this to get more visibility. Despite the extensive work that has been done on hash indexes this release, we have thus far not made any change to the on-disk format that is not nominally backward-compatible. Commit 293e24e507

Re: [HACKERS] WIP: Covering + unique indexes.

2017-03-31 Thread Anastasia Lubennikova
31.03.2017 20:57, Robert Haas: On Fri, Mar 31, 2017 at 1:40 PM, Anastasia Lubennikova wrote: First of all, I want to thank you and Robert for reviewing this patch. Your expertise in postgres subsystems is really necessary for features like this. I just wonder, why don't you share your thoughts

  1   2   >