Re: [HACKERS] logical decoding of two-phase transactions

2020-12-15 Thread Masahiko Sawada
On Mon, Dec 14, 2020 at 6:27 PM Amit Kapila wrote: > > On Tue, Dec 8, 2020 at 2:01 PM Ajin Cherian wrote: > > > > On Tue, Dec 1, 2020 at 6:26 PM Amit Kapila wrote: > > > > > To skip it, we need to send GID in begin message and then on > > > subscriber-side, check if the prepared xact already

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-12-15 Thread Kyotaro Horiguchi
At Tue, 15 Dec 2020 23:10:28 +0900, Fujii Masao wrote in > > I pushed the following two patches. > > - v1-use-standard-SIGHUP-hanlder-in-syslogger-process.patch > > - v1-use-MyLatch-and-standard-SIGHUP-handler-in-startup-process.patch > > As I told in other thread [1], I'm thinking to revert

Refactoring HMAC in the core code

2020-12-15 Thread Michael Paquier
Hi all, (Added Bruce and Daniel in CC:) $subject has been mentioned a couple of times lately, mainly by me for the recent cryptohash refactoring that has been done. We use in the core code HMAC with SHA256 for SCRAM, but this logic should go through SSL libraries able to support it (OpenSSL and

Re: Add Information during standby recovery conflicts

2020-12-15 Thread Kyotaro Horiguchi
At Wed, 16 Dec 2020 14:49:25 +0900 (JST), Kyotaro Horiguchi wrote in > > > > Conflicting processes are 41171, 41194. > > > > Conflicting processes are: 41171, 41194. > > Or I came up with the following after scanning throught the tree. > > | Some processes are conflicting: 41171, 41194. Or

Re: Add Information during standby recovery conflicts

2020-12-15 Thread Kyotaro Horiguchi
At Wed, 16 Dec 2020 12:08:31 +0900, Masahiko Sawada wrote in On Wed, Dec 16, 2020 at 11:22 AM Kyotaro Horiguchi wrote: > > At Tue, 15 Dec 2020 15:40:03 +0900, Fujii Masao wrote in > > > > > > On 2020/12/15 12:04, Kyotaro Horiguchi wrote: > > > [40509:startup] DETAIL: Conflicting processes:

Re: Add Information during standby recovery conflicts

2020-12-15 Thread Masahiko Sawada
On Wed, Dec 16, 2020 at 11:22 AM Kyotaro Horiguchi wrote: > > At Tue, 15 Dec 2020 15:40:03 +0900, Fujii Masao > wrote in > > > > > > On 2020/12/15 12:04, Kyotaro Horiguchi wrote: > > > [40509:startup] DETAIL: Conflicting processes: 41171, 41194. > ... > > > I'm not sure, but it seems to me at

Re: Add Information during standby recovery conflicts

2020-12-15 Thread Masahiko Sawada
On Mon, Dec 14, 2020 at 9:31 PM Fujii Masao wrote: > > > > On 2020/12/05 12:38, Masahiko Sawada wrote: > > On Fri, Dec 4, 2020 at 7:22 PM Drouvot, Bertrand > > wrote: > >> > >> Hi, > >> > >> On 12/4/20 2:21 AM, Fujii Masao wrote: > >>> > >>> On 2020/12/04 9:28, Masahiko Sawada wrote: > On

Re: xid wraparound danger due to INDEX_CLEANUP false

2020-12-15 Thread Masahiko Sawada
On Sat, Nov 21, 2020 at 8:03 AM Peter Geoghegan wrote: > > On Fri, Nov 20, 2020 at 2:17 PM Robert Haas wrote: > > > Does that make sense? > > > > I *think* so. For me the point is that the index never has a right to > > insist on being vacuumed, but it can offer an opinion on how helpful > > it

Re: Misleading comment in prologue of ReorderBufferQueueMessage

2020-12-15 Thread Amit Kapila
On Tue, Dec 15, 2020 at 11:25 AM Ashutosh Bapat wrote: > > On Mon, Dec 14, 2020 at 3:14 PM Amit Kapila wrote: >> >> On Mon, Dec 14, 2020 at 2:45 PM Ashutosh Bapat >> wrote: >> > >> > The name of the function suggests that the given message will be queued in >> > ReorderBuffer. The prologue of

Re: Add Information during standby recovery conflicts

2020-12-15 Thread Kyotaro Horiguchi
At Tue, 15 Dec 2020 15:40:03 +0900, Fujii Masao wrote in > > > On 2020/12/15 12:04, Kyotaro Horiguchi wrote: > > [40509:startup] DETAIL: Conflicting processes: 41171, 41194. ... > > I'm not sure, but it seems to me at least the period is unnecessary > > here. > > Since Error Message Style

Re: archive status ".ready" files may be created too early

2020-12-15 Thread Kyotaro Horiguchi
At Tue, 15 Dec 2020 19:32:57 +0900 (JST), Kyotaro Horiguchi wrote in > At Mon, 14 Dec 2020 18:25:23 +, "Bossart, Nathan" > wrote in > > I wonder if these are safe assumptions to make. For your example, if > > we've written record B to the WAL buffers, but neither record A nor B > > have

Re: pg_shmem_allocations & documentation

2020-12-15 Thread Michael Paquier
On Tue, Dec 15, 2020 at 10:09:35AM +0900, Michael Paquier wrote: > Both of you seem to agree about having more details about that, which > is fine by me at the end. Horiguchi-san, do you have more thoughts to > offer? Benoit's version is similar to yours, just simpler. Okay, applied this one

Re: enable_incremental_sort changes query behavior

2020-12-15 Thread Tomas Vondra
On 12/16/20 1:51 AM, Jaime Casanova wrote: > On Tue, Dec 1, 2020 at 4:08 AM Anastasia Lubennikova > wrote: >> >> On 01.12.2020 03:08, James Coleman wrote: >>> On Tue, Nov 3, 2020 at 4:39 PM Tomas Vondra >>> wrote: I've pushed the 0001 part, i.e. the main fix. Not sure about the other

Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys

2020-12-15 Thread Tomas Vondra
Hi, I reviewed the patch series, tweaked a couple comments, added commit messages etc. Barring objections, I'll push this in a couple days. One thing that annoys me is that it breaks ABI because it adds two new parameters to find_em_expr_usable_for_sorting_rel, but I don't think we can get

Re: Proposed patch for key managment

2020-12-15 Thread Michael Paquier
On Tue, Dec 15, 2020 at 04:02:12PM -0500, Bruce Momjian wrote: > I thought this was going to be a huge job, but once I looked at it, it > was clear exactly what you were saying. Comparing cryptohash.c and > cryptohash_openssl.c I saw exactly what you wanted, and I think I have > completed it in

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-15 Thread Michael Paquier
On Tue, Dec 15, 2020 at 09:45:17PM -0300, Alvaro Herrera wrote: > I don't like this idea too much, because adding an option causes an ABI > break. I don't think we commonly add options in backbranches, but it > has happened. The bitmask is much easier to work with in that regard. ABI

Re: Reloptions for table access methods

2020-12-15 Thread Michael Paquier
On Tue, Dec 15, 2020 at 03:59:02PM -0800, Jeff Davis wrote: > How should that work with the existing code? Should we have separate AM > hooks for each of these interaction points, and then have the AM figure > out how to handle its options? What about the toast.* options? It seems to me that we

Re: enable_incremental_sort changes query behavior

2020-12-15 Thread Jaime Casanova
On Tue, Dec 1, 2020 at 4:08 AM Anastasia Lubennikova wrote: > > On 01.12.2020 03:08, James Coleman wrote: > > On Tue, Nov 3, 2020 at 4:39 PM Tomas Vondra > > wrote: > >> I've pushed the 0001 part, i.e. the main fix. Not sure about the other > >> parts (comments and moving the code back to

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-15 Thread Alvaro Herrera
On 2020-Dec-12, Peter Eisentraut wrote: > On 2020-12-11 21:27, Alvaro Herrera wrote: > > By the way-- What did you think of the idea of explictly marking the > > types used for bitmasks using types bits32 and friends, instead of plain > > int, which is harder to spot? > > If we want to make it

Re: Proposed patch for key managment

2020-12-15 Thread Michael Paquier
Hi Stephen, On Tue, Dec 15, 2020 at 02:09:09PM -0500, Stephen Frost wrote: > Yeah, looking at what's been done there seems like the right approach to > me as well, ideally we'd have one set of APIs that'll support all these > use cases (not unlike what curl does..). Ooh... This is interesting.

Re: Minor documentation error regarding streaming replication protocol

2020-12-15 Thread Michael Paquier
On Tue, Dec 15, 2020 at 12:54:51PM -0800, Andres Freund wrote: > Minor nit: I'd put this into common/string.[ch], besides > pg_clean_ascii(). Probably renaming it to pg_is_ascii(). Yeah. There is already one pg_is_ascii_string() in saslprep.c, is_all_ascii() in collationcmds.c and

Re: REINDEX backend filtering

2020-12-15 Thread Michael Paquier
On Tue, Dec 15, 2020 at 06:34:16PM +0100, Magnus Hagander wrote: > Is this really a common enough operation that we need it in the main grammar? > > Having the functionality, definitely, but what if it was "just" a > function instead? So you'd do something like: > SELECT 'reindex index ' || i

Re: copy.sgml and partitioned tables

2020-12-15 Thread Bruce Momjian
On Thu, Dec 3, 2020 at 03:17:23PM -0600, Justin Pryzby wrote: > https://www.postgresql.org/docs/current/sql-copy.html > |. COPY FROM can be used with plain, foreign, or partitioned tables or with > views that have INSTEAD OF INSERT triggers. > |. COPY only deals with the specific table named; IT

Re: Proposed patch for key managment

2020-12-15 Thread Bruce Momjian
On Tue, Dec 15, 2020 at 11:13:12PM +, Alastair Turner wrote: > Since it's exciting stuff, I've been trying to lash together a PoC integration > with the crypto infrastructure we see at these customers. Unfortunately, in > short, the API doesn't seem to suit integration with HSM big iron, like

Re: Reloptions for table access methods

2020-12-15 Thread Jeff Davis
On Tue, 2020-12-15 at 17:37 +, Simon Riggs wrote: > > I definitely don't agree with the premise that all existing heap > options should be common across all new or extension tableAMs. There > are dozens of such options and I don't believe that they would all > have meaning in all future

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-15 Thread Justin Pryzby
On Mon, Dec 14, 2020 at 06:14:17PM -0600, Justin Pryzby wrote: > On Sat, Dec 12, 2020 at 01:45:26PM -0600, Justin Pryzby wrote: > > On Sat, Dec 12, 2020 at 09:20:35AM +0100, Peter Eisentraut wrote: > > > On 2020-12-11 21:27, Alvaro Herrera wrote: > > > > By the way-- What did you think of the

Re: Sorting case branches in outfuncs.c/outNode alphabetically

2020-12-15 Thread Tom Lane
Fedir Panasenko writes: > outfuncs.c contains a switch statement responsible for choosing > serialization function per node type here: > https://github.com/postgres/postgres/blob/master/src/backend/nodes/outfuncs.c#L3711 Why are you concerned about outfuncs.c in particular? Its sibling files

Re: Proposed patch for key managment

2020-12-15 Thread Alastair Turner
Hi Bruce et al Firstly, thanks for shaping the patch, getting it down to a manageable scope of cluster file encryption. I think this is a great feature and it matters to a lot of the customers I talk to at VMware about adopting Postgres. Since it's exciting stuff, I've been trying to lash

Sorting case branches in outfuncs.c/outNode alphabetically

2020-12-15 Thread Fedir Panasenko
Hi all, outfuncs.c contains a switch statement responsible for choosing serialization function per node type here: https://github.com/postgres/postgres/blob/master/src/backend/nodes/outfuncs.c#L3711 It spans over >650LOC and is quite unreadable, requiring using search or code analysis tools for

Re: Minor documentation error regarding streaming replication protocol

2020-12-15 Thread Bruce Momjian
On Tue, Dec 15, 2020 at 01:22:44AM -0800, Jeff Davis wrote: > On Thu, 2020-12-03 at 13:14 -0500, Bruce Momjian wrote: > > Andres objected (in a separate conversation) to forcing a binary- > > > format > > > value on a client that didn't ask for one. He suggested that we > > > mandate > > > that

Re: SELECT INTO deprecation

2020-12-15 Thread Bruce Momjian
On Wed, Dec 9, 2020 at 09:48:54PM +0100, Peter Eisentraut wrote: > On 2020-12-03 20:26, Peter Eisentraut wrote: > > On 2020-12-03 16:34, Tom Lane wrote: > > > As I recall, a whole lot of the pain we have with INTO has to do > > > with the semantics we've chosen for INTO in a set-operation nest. >

Re: pg_rewind copies

2020-12-15 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested Hello The patch seems to do as described and the regression and

Re: Proposed patch for key managment

2020-12-15 Thread Bruce Momjian
On Tue, Dec 15, 2020 at 02:09:09PM -0500, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Dec 15, 2020 at 10:05:49AM +0900, Michael Paquier wrote: > > > On Mon, Dec 14, 2020 at 06:06:15PM -0500, Bruce Momjian wrote: > > > > I am getting close to

Re: Proposed patch for key managment

2020-12-15 Thread Bruce Momjian
On Tue, Dec 15, 2020 at 02:20:33PM +0900, Michael Paquier wrote: > On Mon, Dec 14, 2020 at 10:19:02PM -0500, Bruce Momjian wrote: > > I am going to need someone to help me make these changes. I don't feel > > I know enough about the crypto API to do it, and it will take me 1+ week > > to learn

Re: Minor documentation error regarding streaming replication protocol

2020-12-15 Thread Andres Freund
Hi, On 2020-12-15 01:22:44 -0800, Jeff Davis wrote: > Attached a simple patch that enforces an all-ASCII restore point name > rather than documenting the possibility of non-ASCII text. +1 > diff --git a/src/backend/access/transam/xlogfuncs.c > b/src/backend/access/transam/xlogfuncs.c > index

Re: Rethinking plpgsql's assignment implementation

2020-12-15 Thread Tom Lane
I realized that the speedup patch I posted yesterday is flawed: it's too aggressive about applying the R/W param mechanism, instead of not aggressive enough. To review, the point of that logic is that if we have an assignment like arrayvar := array_append(arrayvar,

Re: SQL/JSON: functions

2020-12-15 Thread Pavel Stehule
út 15. 12. 2020 v 19:56 odesílatel Oleg Bartunov napsal: > On Tue, Dec 15, 2020 at 8:37 PM Pavel Stehule > wrote: > > > > > > > > út 15. 12. 2020 v 18:00 odesílatel Simon Riggs > napsal: > >> > >> On Fri, 17 Jul 2020 at 21:26, Nikita Glukhov > wrote: > >> > > >> > Attached 50th version of the

Re: SQL/JSON: functions

2020-12-15 Thread Oleg Bartunov
On Tue, Dec 15, 2020 at 8:37 PM Pavel Stehule wrote: > > > > út 15. 12. 2020 v 18:00 odesílatel Simon Riggs napsal: >> >> On Fri, 17 Jul 2020 at 21:26, Nikita Glukhov wrote: >> > >> > Attached 50th version of the patches. Only the documentation was changed >> > since the previous version. >> >>

Re: SQL/JSON: functions

2020-12-15 Thread Oleg Bartunov
On Tue, Dec 15, 2020 at 8:01 PM Simon Riggs wrote: > > On Fri, 17 Jul 2020 at 21:26, Nikita Glukhov wrote: > > > > Attached 50th version of the patches. Only the documentation was changed > > since the previous version. > > I can imagine the effort required to get to v50, so I salute your

Re: Proposed patch for key managment

2020-12-15 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Dec 15, 2020 at 10:05:49AM +0900, Michael Paquier wrote: > > On Mon, Dec 14, 2020 at 06:06:15PM -0500, Bruce Momjian wrote: > > > I am getting close to applying these patches, probably this week. The > > > patches are cumulative: > >

Re: Minimal logical decoding on standbys

2020-12-15 Thread Fabrízio de Royes Mello
On Wed, Mar 18, 2020 at 4:50 PM Alvaro Herrera wrote: > > > well, not "forever", but: > > $ make check PROVE_TESTS=t/019_standby_logical_decoding_conflicts.pl PROVE_FLAGS=-v > ... > cd /pgsql/source/master/src/test/recovery &&

Re: Reloptions for table access methods

2020-12-15 Thread Simon Riggs
On Tue, 1 Sept 2020 at 18:21, Jeff Davis wrote: > I went with the simple approach because fixing that problem seemed a > bit over-engineered. Here are some thoughts on what we could do: The simple patch is admirable, but not something we should put into core. I definitely don't agree with the

Re: SQL/JSON: functions

2020-12-15 Thread Pavel Stehule
út 15. 12. 2020 v 18:00 odesílatel Simon Riggs napsal: > On Fri, 17 Jul 2020 at 21:26, Nikita Glukhov > wrote: > > > > Attached 50th version of the patches. Only the documentation was changed > > since the previous version. > > I can imagine the effort required to get to v50, so I salute your

Re: REINDEX backend filtering

2020-12-15 Thread Magnus Hagander
On Tue, Dec 15, 2020 at 12:22 PM Julien Rouhaud wrote: > > On Mon, Dec 14, 2020 at 3:45 PM Michael Paquier wrote: > > > > On Thu, Dec 03, 2020 at 05:31:43PM +0800, Julien Rouhaud wrote: > > > Now that we have the infrastructure to track indexes that might be > > > corrupted > > > due to changes

Re: On login trigger: take three

2020-12-15 Thread Pavel Stehule
> Please notice that we still need GUC to disable on-login triggers: to make >> it possible for superuser who did mistake and defined incorrect on-login >> trigger to >> login to the system. >> Do we need GUC to disable all other event triggers? May be I am wrong, >> but I do not see much need in

Re: SQL/JSON: functions

2020-12-15 Thread Simon Riggs
On Fri, 17 Jul 2020 at 21:26, Nikita Glukhov wrote: > > Attached 50th version of the patches. Only the documentation was changed > since the previous version. I can imagine the effort required to get to v50, so I salute your efforts. The document for SQL Standard has now been published as CD

Re: Proposed patch for key managment

2020-12-15 Thread Bruce Momjian
On Tue, Dec 15, 2020 at 10:36:56AM +0800, Neil Chen wrote: > 2. I tried to add support for AES_CTR mode, and the code for encrypting buffer > blocks. In the process I found that in pg_cipher_ctx_create, the key length is > declared as "byte". However, in the CryptoKey structure, the length is

Re: On login trigger: take three

2020-12-15 Thread Konstantin Knizhnik
On 15.12.2020 18:25, Pavel Stehule wrote: út 15. 12. 2020 v 15:06 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> napsal: On 15.12.2020 16:18, Pavel Stehule wrote: út 15. 12. 2020 v 14:12 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>>

Re: Proposed patch for key managment

2020-12-15 Thread Bruce Momjian
On Mon, Dec 14, 2020 at 11:16:18PM -0500, Bruce Momjian wrote: > > 1. Previously, we added a variable bootstrap_keys_wrap that is used for > > encryption during initdb. However, since we save the "wrapped" key, we need > > to > > use a global KEK that can be accessed in boot mode to unwrap it

Re: On login trigger: take three

2020-12-15 Thread Pavel Stehule
út 15. 12. 2020 v 15:06 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > > > On 15.12.2020 16:18, Pavel Stehule wrote: > > > > út 15. 12. 2020 v 14:12 odesílatel Konstantin Knizhnik < > k.knizh...@postgrespro.ru> napsal: > >> >> >> On 11.12.2020 19:27, Pavel Stehule wrote: >>

Re: Proposed patch for key managment

2020-12-15 Thread Bruce Momjian
On Tue, Dec 15, 2020 at 02:20:33PM +0900, Michael Paquier wrote: > > Uh, I got this code from pg_resetwal.c, which does something similar to > > pg_altercpass. > > Yeah, that's actually the part where it is a bad idea to just copy > this pattern. pg_resetwal should not do that in the long term

Re: Speeding up GIST index creation for tsvectors

2020-12-15 Thread Amit Khandekar
On Sun, 13 Dec 2020 at 9:28 PM, Andrey Borodin wrote: > +1 > This will make all INSERTs and UPDATES for tsvector's GiSTs. Oh, I didn't realize that this code is getting used in GIST index insertion and creation too. Will check there. > Also I really like idea of taking advantage of hardware

Re: HASH_BLOBS hazards (was Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions)

2020-12-15 Thread Tom Lane
Noah Misch writes: > On Mon, Dec 14, 2020 at 01:59:03PM -0500, Tom Lane wrote: >> Here's a rolled-up patch that does some further documentation work >> and gets rid of the unnecessary memset's as well. > +1 on removing the memset() calls. That said, it's not a big deal if more > creep in over

Re: On login trigger: take three

2020-12-15 Thread Pavel Stehule
út 15. 12. 2020 v 15:06 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > > > On 15.12.2020 16:18, Pavel Stehule wrote: > > > > út 15. 12. 2020 v 14:12 odesílatel Konstantin Knizhnik < > k.knizh...@postgrespro.ru> napsal: > >> >> >> On 11.12.2020 19:27, Pavel Stehule wrote: >>

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-12-15 Thread Fujii Masao
On 2020/11/04 18:06, Fujii Masao wrote: On 2020/10/29 15:21, Fujii Masao wrote: On 2020/10/07 11:30, Bharath Rupireddy wrote: On Tue, Oct 6, 2020 at 11:41 AM Bharath Rupireddy wrote: On Tue, Oct 6, 2020 at 11:20 AM Fujii Masao wrote: +1 Or it's also ok to make each patch

Re: On login trigger: take three

2020-12-15 Thread Konstantin Knizhnik
On 15.12.2020 16:18, Pavel Stehule wrote: út 15. 12. 2020 v 14:12 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> napsal: On 11.12.2020 19:27, Pavel Stehule wrote: pá 11. 12. 2020 v 17:05 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>>

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2020-12-15 Thread Peter Eisentraut
Here is a new patch for this. This now follows the implementation that Tom has suggested: Leave date_part() alone, add a new set of extract() functions, and map the SQL EXTRACT construct to those. I have basically just copied over the implementations from my previous patch and placed them

Re: TAP tests aren't using the magic words for Windows file access

2020-12-15 Thread Andrew Dunstan
On 12/15/20 12:05 AM, r.zhar...@postgrespro.ru wrote: > Hello hackers, > > Are there any plans to backport the patch to earlier versions > of the Postgres? > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=114541d58e5970e51b78b77b65de16210beaab43 > > > We rarely see the issue with

Re: On login trigger: take three

2020-12-15 Thread Pavel Stehule
út 15. 12. 2020 v 14:12 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > > > On 11.12.2020 19:27, Pavel Stehule wrote: > > > > pá 11. 12. 2020 v 17:05 odesílatel Konstantin Knizhnik < > k.knizh...@postgrespro.ru> napsal: > >> >> >> On 11.12.2020 18:40, Pavel Stehule wrote: >>

Re: On login trigger: take three

2020-12-15 Thread Konstantin Knizhnik
On 11.12.2020 19:27, Pavel Stehule wrote: pá 11. 12. 2020 v 17:05 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> napsal: On 11.12.2020 18:40, Pavel Stehule wrote: is not correct. It makes it not possible to superuser to disable triggers for all

Re: Movement of restart_lsn position movement of logical replication slots is very slow

2020-12-15 Thread Amit Kapila
On Tue, Dec 15, 2020 at 11:00 AM Jammie wrote: > > Thanks Amit for the response > > We are using pgJDBC sample program here > https://jdbc.postgresql.org/documentation/head/replication.html > > the setFlushLSN is coming from the pgJDBC only. > > git hub for APIs of pgJDBC methods available. > >

Re: Add session statistics to pg_stat_database

2020-12-15 Thread Laurenz Albe
On Tue, 2020-12-15 at 13:53 +0100, Laurenz Albe wrote: > Attached is patch version 9. Aah, I forgot the ++. Version 10 attached. Yours, Laurenz Albe From b40e34141c80ff59c0005f430bd8c273918eb7bb Mon Sep 17 00:00:00 2001 From: Laurenz Albe Date: Tue, 15 Dec 2020 13:46:44 +0100 Subject: [PATCH]

Re: Add session statistics to pg_stat_database

2020-12-15 Thread Laurenz Albe
On Sun, 2020-12-13 at 17:49 +0100, Magnus Hagander wrote: > > > I am considering the cases > > > > > > 1) client just went away (currently "aborted") > > > 2) death by FATAL error > > > 3) killed by the administrator (or shutdown) > > > > I named the three counters "sessions_client_eof",

Re: [HACKERS] logical decoding of two-phase transactions

2020-12-15 Thread Amit Kapila
On Mon, Dec 14, 2020 at 2:59 PM Amit Kapila wrote: > Today, I looked at one of the issues discussed earlier in this thread [1] which is that decoding can block (or deadlock can happen) when the user explicitly locks the catalog relation (like Lock pg_class) or perform Cluster on non-relmapped

Re: Parallel Inserts in CREATE TABLE AS

2020-12-15 Thread Bharath Rupireddy
On Tue, Dec 15, 2020 at 5:48 PM Hou, Zhijie wrote: > > A little explanation about how to push down the ctas info in append. > > > > 1. about how to ignore tuple cost in this case. > > IMO, it create gather path under append like the following: > > query_planner > > -make_one_rel > >

RE: Parallel Inserts in CREATE TABLE AS

2020-12-15 Thread Hou, Zhijie
> From: Hou, Zhijie [mailto:houzj.f...@cn.fujitsu.com] > Sent: Tuesday, December 15, 2020 7:30 PM > To: Bharath Rupireddy > Cc: Amit Kapila ; Luc Vlaming ; > PostgreSQL-development ; Zhihong Yu > ; Dilip Kumar > Subject: RE: Parallel Inserts in CREATE TABLE AS > > > Thanks for the append use

RE: libpq debug log

2020-12-15 Thread iwata....@fujitsu.com
Hi Alvaro san, > There are some things still to do: I worked on some to do. > 1. Is the handling of protocol 2 correct? Protocol 3.0 is implemented in PostgreSQL v7.4 or later. Therefore, most servers and clients today want to connect using 3.0. Also, wishful thinking, I think Protocol 2.0

RE: Parallel Inserts in CREATE TABLE AS

2020-12-15 Thread Hou, Zhijie
> Thanks for the append use case. > > Here's my analysis on pushing parallel inserts down even in case the top > node is Append. > > For union cases which need to remove duplicate tuples, we can't push the > inserts or CTAS dest receiver down. If I'm not wrong, Append node is not > doing

Re: REINDEX backend filtering

2020-12-15 Thread Julien Rouhaud
On Mon, Dec 14, 2020 at 3:45 PM Michael Paquier wrote: > > On Thu, Dec 03, 2020 at 05:31:43PM +0800, Julien Rouhaud wrote: > > Now that we have the infrastructure to track indexes that might be corrupted > > due to changes in collation libraries, I think it would be a good idea to > > offer > >

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-15 Thread Alexey Kondratov
On 2020-12-15 03:14, Justin Pryzby wrote: On Sat, Dec 12, 2020 at 01:45:26PM -0600, Justin Pryzby wrote: On Sat, Dec 12, 2020 at 09:20:35AM +0100, Peter Eisentraut wrote: > On 2020-12-11 21:27, Alvaro Herrera wrote: > > By the way-- What did you think of the idea of explictly marking the > >

Re: archive status ".ready" files may be created too early

2020-12-15 Thread Kyotaro Horiguchi
At Mon, 14 Dec 2020 18:25:23 +, "Bossart, Nathan" wrote in > Apologies for the long delay. > > I've spent a good amount of time thinking about this bug and trying > out a few different approaches for fixing it. I've attached a work- > in-progress patch for my latest attempt. > > On

Re: Single transaction in the tablesync worker?

2020-12-15 Thread Peter Smith
design may be needed for this part. * help / comments / cleanup * There is temporary "!!>>" excessive logging of mine scattered around which I added to help my testing during development --- Kind Regards, Peter Smith. Fujitsu Australia v3-0002-2PC-Solution1-WIP-20201215.patch De

Re: Minor documentation error regarding streaming replication protocol

2020-12-15 Thread Jeff Davis
On Thu, 2020-12-03 at 13:14 -0500, Bruce Momjian wrote: > Andres objected (in a separate conversation) to forcing a binary- > > format > > value on a client that didn't ask for one. He suggested that we > > mandate > > that the data is ASCII-only (for both filename and content), > > closing > >

Re: Parallel Inserts in CREATE TABLE AS

2020-12-15 Thread Dilip Kumar
On Tue, Dec 15, 2020 at 2:06 PM Bharath Rupireddy wrote: > > On Mon, Dec 14, 2020 at 6:08 PM Hou, Zhijie wrote: > > Currently with the patch, we can allow parallel CTAS when topnode is Gather. > > When top-node is Append and Gather is the sub-node of Append, I think we > > can still enable > >

Re: libpq debug log

2020-12-15 Thread Greg Nancarrow
On Tue, Oct 27, 2020 at 3:23 AM Alvaro Herrera wrote: > > I've been hacking at this patch again. There were a few things I wasn't > too clear about, so I reordered the code and renamed the routines to try > to make it easier to follow. > Hi, Hopefully Iwata-san will return to looking at this

Re: Parallel Inserts in CREATE TABLE AS

2020-12-15 Thread Bharath Rupireddy
On Mon, Dec 14, 2020 at 6:08 PM Hou, Zhijie wrote: > Currently with the patch, we can allow parallel CTAS when topnode is Gather. > When top-node is Append and Gather is the sub-node of Append, I think we can > still enable > Parallel CTAS by pushing Parallel CTAS down to the sub-node Gather,

Re: TAP tests aren't using the magic words for Windows file access

2020-12-15 Thread r . zharkov
Hello hackers, Are there any plans to backport the patch to earlier versions of the Postgres? https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=114541d58e5970e51b78b77b65de16210beaab43 We rarely see the issue with the pg_ctl/004_logrotate test on the REL_12_STABLE branch. On my

Re: HASH_BLOBS hazards (was Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions)

2020-12-15 Thread Noah Misch
On Mon, Dec 14, 2020 at 01:59:03PM -0500, Tom Lane wrote: > * Should we just have a blanket insistence that all callers supply > HASH_ELEM? The default sizes that dynahash.c uses without that are > undocumented and basically useless. +1 > we should just rip out all those memsets as pointless,