Re: Parallel Inserts in CREATE TABLE AS

2020-10-15 Thread Bharath Rupireddy
On Fri, Oct 16, 2020 at 11:33 AM Luc Vlaming wrote: > > Really looking forward to this ending up in postgres as I think it's a > very nice improvement. > > Whilst reviewing your patch I was wondering: is there a reason you did > not introduce a batch insert in the destreceiver for the CTAS? For me

Re: Parallel Inserts in CREATE TABLE AS

2020-10-15 Thread Luc Vlaming
On 14.10.20 11:16, Bharath Rupireddy wrote: On Tue, Oct 6, 2020 at 10:58 AM Amit Kapila wrote: Yes we do a bunch of catalog changes related to the created new table. We will have both the txn id and command id assigned when catalogue changes are being made. But, right after the table is creat

Re: upcoming API changes for LLVM 12

2020-10-15 Thread Alvaro Herrera
On 2020-Oct-15, Andres Freund wrote: > There will be a breaking API change for JIT related API in LLVM > 12. Mostly about making control over various aspects easier, and then > building on top of that providing new features (like JIT compiling in > the background and making it easier to share JIT

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-15 Thread Amit Kapila
On Thu, Oct 15, 2020 at 6:13 PM Amit Kapila wrote: > > On Thu, Oct 15, 2020 at 9:56 AM Greg Nancarrow wrote: > > > > Also, I'm seeing a partition-related error when running > > installcheck-world - I'm investigating that. > > > > Okay. > The attached patch fixes this partition case for me. Basic

RE: Use list_delete_xxxcell O(1) instead of list_delete_ptr O(N) in some places

2020-10-15 Thread Hou, Zhijie
> > I found some code places call list_delete_ptr can be replaced by > list_delete_xxxcell which can be faster. > > > > diff --git a/src/backend/optimizer/path/joinpath.c > > b/src/backend/optimizer/path/joinpath.c > > index db54a6b..61ef7c8 100644 > > --- a/src/backend/optimizer/path/joinpath.c >

Re: speed up unicode decomposition and recomposition

2020-10-15 Thread Michael Paquier
On Thu, Oct 15, 2020 at 01:59:38PM -0400, John Naylor wrote: > I think I've seen a trie recommended somewhere, maybe the official website. > That said, I was able to get the hash working for recomposition (split into > a separate patch, and both of them now leave frontend alone), and I'm > pleased

Re: gs_group_1 crashing on 13beta2/s390x

2020-10-15 Thread Andres Freund
Hi, On 2020-10-15 17:12:54 -0700, Andres Freund wrote: > On 2020-10-15 15:37:01 -0700, Andres Freund wrote: > It's a bug that was fixed in LLVM 4, but too late to be backported to > 3.9. > > The easiest seems to be to just use a wrapper function that does the > necessary pre-checks. Something lik

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-15 Thread Amit Kapila
On Fri, Oct 16, 2020 at 2:00 AM Andres Freund wrote: > > On 2020-10-15 12:38:49 +0530, Amit Kapila wrote: > > On Wed, Oct 14, 2020 at 4:51 PM Dilip Kumar wrote: > > > > > > On Wed, Oct 14, 2020 at 4:12 PM Amit Kapila > > > wrote: > > > > > > > > > > > > Thanks for the tests. The latest patch lo

Re: upcoming API changes for LLVM 12

2020-10-15 Thread Thomas Munro
On Fri, Oct 16, 2020 at 2:12 PM Andres Freund wrote: > There will be a breaking API change for JIT related API in LLVM > 12. Mostly about making control over various aspects easier, and then > building on top of that providing new features (like JIT compiling in > the background and making it easi

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2020-10-15 Thread Craig Ringer
On Fri, 16 Oct 2020, 09:00 Michael Paquier, wrote: > On Thu, Oct 15, 2020 at 01:06:54PM -0400, Tom Lane wrote: > > Other than src/test/modules/brin, the ISOLATION users don't look > > much like real extensions (rather than test scaffolding), either. > > If you discount test scaffolding modules th

Re: Online checksums verification in the backend

2020-10-15 Thread Julien Rouhaud
On Fri, Oct 16, 2020 at 8:59 AM Julien Rouhaud wrote: > > On Thu, Oct 15, 2020 at 3:59 PM Julien Rouhaud wrote: > > > > On Thu, Oct 15, 2020 at 1:37 PM Julien Rouhaud wrote: > > > > > > On Thu, Oct 1, 2020 at 1:07 PM Michael Paquier > > > wrote: > > > > > > > > On Fri, Sep 25, 2020 at 06:11:47

upcoming API changes for LLVM 12

2020-10-15 Thread Andres Freund
Hi, There will be a breaking API change for JIT related API in LLVM 12. Mostly about making control over various aspects easier, and then building on top of that providing new features (like JIT compiling in the background and making it easier to share JIT compiled output between processes). I've

Re: Is the doc for SHOW somewhat stale?

2020-10-15 Thread Kyotaro Horiguchi
Thank you for the comments. At Thu, 15 Oct 2020 23:18:33 +0900, Fujii Masao wrote in > > > On 2020/10/06 16:53, Kyotaro Horiguchi wrote: > > Hello. > > If I read the documentation of the SHOW command, it says: > > https://www.postgresql.org/docs/13/sql-show.html > > > >> name > >>The nam

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2020-10-15 Thread Michael Paquier
On Thu, Oct 15, 2020 at 01:06:54PM -0400, Tom Lane wrote: > Other than src/test/modules/brin, the ISOLATION users don't look > much like real extensions (rather than test scaffolding), either. > If you discount test scaffolding modules then the use-counts are > more like 4 to 1. Out of core, the o

Re: Online checksums verification in the backend

2020-10-15 Thread Julien Rouhaud
On Thu, Oct 15, 2020 at 3:59 PM Julien Rouhaud wrote: > > On Thu, Oct 15, 2020 at 1:37 PM Julien Rouhaud wrote: > > > > On Thu, Oct 1, 2020 at 1:07 PM Michael Paquier wrote: > > > > > > On Fri, Sep 25, 2020 at 06:11:47PM +0800, Julien Rouhaud wrote: > > > > Thanks a lot for the tests! I'm not s

Re: pg_upgrade: fail early if a tablespace dir already exists for new cluster version

2020-10-15 Thread Bruce Momjian
On Thu, Oct 15, 2020 at 05:19:59PM -0700, Andres Freund wrote: > Hi, > > On 2020-10-15 19:35:30 -0400, Bruce Momjian wrote: > > On Fri, Oct 9, 2020 at 07:42:51PM -0400, Bruce Momjian wrote: > > > On Fri, Oct 9, 2020 at 02:23:10PM -0500, Justin Pryzby wrote: > > > > In my local branch, I had revi

Re: pg_upgrade: fail early if a tablespace dir already exists for new cluster version

2020-10-15 Thread Andres Freund
Hi, On 2020-10-15 19:35:30 -0400, Bruce Momjian wrote: > On Fri, Oct 9, 2020 at 07:42:51PM -0400, Bruce Momjian wrote: > > On Fri, Oct 9, 2020 at 02:23:10PM -0500, Justin Pryzby wrote: > > > In my local branch, I had revised this comment to say: > > > > > > + * Note, v8.4 has no tablespace_suff

Re: gs_group_1 crashing on 13beta2/s390x

2020-10-15 Thread Andres Freund
Hi, On 2020-10-15 15:37:01 -0700, Andres Freund wrote: > On 2020-10-15 15:29:24 -0700, Andres Freund wrote: > > Pushed now to 11-master. > > Ugh - there's a failure with an old LLVM version (3.9): > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dragonet&dt=2020-10-15%2022%3A24%3A04 > >

Re: Some remaining htonl() and ntohl() calls in the code

2020-10-15 Thread Michael Paquier
On Wed, Oct 14, 2020 at 01:41:23PM -0700, Andres Freund wrote: > +1 Thanks. I have applied that as of 86dba33, and did not see a need for a back-patch. -- Michael signature.asc Description: PGP signature

Re: Wrong example in the bloom documentation

2020-10-15 Thread Bruce Momjian
On Fri, Oct 9, 2020 at 11:08:32AM -0400, Bruce Momjian wrote: > On Fri, Oct 9, 2020 at 05:44:57AM +, Daniel Westermann (DWE) wrote: > > Hi Bruce, Tom, > > > > On Thu, Oct  8, 2020 at 03:43:32PM -0400, Tom Lane wrote: > > >> "Daniel Westermann (DWE)" writes: > > >> >> I was hoping someone mo

Re: pg_upgrade: fail early if a tablespace dir already exists for new cluster version

2020-10-15 Thread Bruce Momjian
On Fri, Oct 9, 2020 at 07:42:51PM -0400, Bruce Momjian wrote: > On Fri, Oct 9, 2020 at 02:23:10PM -0500, Justin Pryzby wrote: > > In my local branch, I had revised this comment to say: > > > > + * Note, v8.4 has no tablespace_suffix, which is fine so long as the > > version we > > + * being upg

Re: jit and explain nontext

2020-10-15 Thread Andres Freund
Hi, On 2020-10-15 14:51:38 +1300, David Rowley wrote: > That's a pretty good point. If we do SET enable_sort TO off; then > cached plans are unaffected. In contrast to that we do however use the current work_mem for execution, I believe. Which could be fairly nasty, if a plan was made for a huge

Re: gs_group_1 crashing on 13beta2/s390x

2020-10-15 Thread Andres Freund
Hi, On 2020-10-15 15:29:24 -0700, Andres Freund wrote: > Pushed now to 11-master. Ugh - there's a failure with an old LLVM version (3.9): https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dragonet&dt=2020-10-15%2022%3A24%3A04 Need to rebuild that locally to reproduce. Greetings, Andres F

Re: gs_group_1 crashing on 13beta2/s390x

2020-10-15 Thread Andres Freund
Hi, On 2020-10-15 01:32:46 -0700, Andres Freund wrote: > I have a fix for this, but I've just stared at s390 assembly code for > ~10h, never having done so before. So that'll have to wait for tomorrow. > > It's quite possible that that fix would also help on other > architectures... Pushed now to

Re: Use list_delete_xxxcell O(1) instead of list_delete_ptr O(N) in some places

2020-10-15 Thread David Rowley
On Sat, 10 Oct 2020 at 15:45, Hou, Zhijie wrote: > I found some code places call list_delete_ptr can be replaced by > list_delete_xxxcell which can be faster. > > diff --git a/src/backend/optimizer/path/joinpath.c > b/src/backend/optimizer/path/joinpath.c > index db54a6b..61ef7c8 100644 > --- a/

Re: stress test for parallel workers

2020-10-15 Thread Thomas Munro
On Tue, Aug 25, 2020 at 1:43 AM Tom Lane wrote: > I wrote: > > For our archives' sake: today I got seemingly-automated mail informing me > > that this patch has been merged into the 4.19-stable, 5.4-stable, > > 5.7-stable, and 5.8-stable kernel branches; but not 4.4-stable, > > 4.9-stable, or 4.14

Re: kevent latch paths don't handle postmaster death well

2020-10-15 Thread Tom Lane
Thomas Munro writes: > I couldn't resist digging further into the Apple sources to figure out > what was going on there, and I realised that the code path I was > looking at can only report EACCES if you asked for NOTE_EXITSTATUS, > which appears to be an Apple extension to the original FreeBSD kq

Re: kevent latch paths don't handle postmaster death well

2020-10-15 Thread Thomas Munro
On Thu, Oct 15, 2020 at 6:42 PM Thomas Munro wrote: > I tried to test this on my system but it seems like maybe FreeBSD > can't really report EACCES for EVFILT_PROC. From the man page and a > quick inspection of the source, you only have to be able to "see" the > process, and if you can't I think

Re: [Patch] Using Windows groups for SSPI authentication

2020-10-15 Thread Russell Foster
On Thu, Oct 15, 2020 at 11:31 AM Stephen Frost wrote: > Please don't top-post on these lists.. Didn't even know what that was, had to look it up. Hopefully it is resolved. Gmail does too many things for you! > While not exactly the same, of course, they are more-or-less equivilant > to Unix grou

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-15 Thread Andres Freund
On 2020-10-15 12:38:49 +0530, Amit Kapila wrote: > On Wed, Oct 14, 2020 at 4:51 PM Dilip Kumar wrote: > > > > On Wed, Oct 14, 2020 at 4:12 PM Amit Kapila wrote: > > > > > > > > > Thanks for the tests. The latest patch looks mostly good to me. I have > > > made minor changes to the patch (a) chang

Re: Partition prune with stable Expr

2020-10-15 Thread Andy Fan
On Mon, Sep 28, 2020 at 8:21 PM Andy Fan wrote: > > > On Mon, Sep 28, 2020 at 7:15 AM Tom Lane wrote: > >> Andy Fan writes: >> > On Mon, Sep 28, 2020 at 4:46 AM David Rowley >> wrote: >> >> Thanks for showing an interest in partition pruning. Unfortunately, >> >> it's not possible to use stabl

Re: improve the algorithm cached_plan_cost with real planning time?

2020-10-15 Thread Andy Fan
On Thu, Oct 15, 2020 at 9:12 PM Andy Fan wrote: > > In cached_plan_cost, we do consider the cost of planning, with the > following > algorithm. > > int nrelations = list_length(plannedstmt->rtable); > > result += 1000.0 * cpu_operator_cost * (nrelations + 1); > > I run into a case where 10 relati

Re: plan cache doesn't clean plans with references to dropped procedures

2020-10-15 Thread Pavel Stehule
čt 15. 10. 2020 v 20:17 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > I am playing with fixing the speed of CALL statement in a non atomic > > context, and when I tested my patch I found another issue of CALL > statement > > - an invalidation of plans doesn't work for CALL statement (i

Re: plan cache doesn't clean plans with references to dropped procedures

2020-10-15 Thread Tom Lane
Pavel Stehule writes: > I am playing with fixing the speed of CALL statement in a non atomic > context, and when I tested my patch I found another issue of CALL statement > - an invalidation of plans doesn't work for CALL statement (in atomic > context). Yeah, that's not the plancache's fault. C

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2020-10-15 Thread Justin Pryzby
I'm hoping that Alvaro will comment on this. On Wed, Sep 30, 2020 at 05:34:50PM -0500, Justin Pryzby wrote: > CREATE TABLE t(i int) PARTITION BY RANGE(i); > CREATE TABLE t1 PARTITION OF t FOR VALUES FROM (1) TO (10); > CREATE OR REPLACE FUNCTION tgf() RETURNS trigger LANGUAGE plpgsql AS $$ begin

plan cache doesn't clean plans with references to dropped procedures

2020-10-15 Thread Pavel Stehule
Hi I am playing with fixing the speed of CALL statement in a non atomic context, and when I tested my patch I found another issue of CALL statement - an invalidation of plans doesn't work for CALL statement (in atomic context). CREATE OR REPLACE FUNCTION public.fx(a integer) RETURNS integer LAN

Re: AppendStringInfoChar instead of appendStringInfoString

2020-10-15 Thread Alvaro Herrera
On 2020-Sep-27, Justin Pryzby wrote: > On Fri, Sep 25, 2020 at 08:49:57AM +, Hou, Zhijie wrote: > > In (/src/backend/replication/backup_manifest.c) > > > > I found the following code: > > > > appendStringInfoString(&buf, "\n"); > > appendStringInfoString(&buf, "\""); > > Good point.

psql \df choose functions by their arguments

2020-10-15 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Improve psql \df to choose functions by their arguments == OVERVIEW Having to scroll through same-named functions with different argument types when you know exactly which one you want is annoying at best, error causing at worst. This patch ena

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2020-10-15 Thread Tom Lane
Alvaro Herrera writes: > I forgot to mention that I considered backpatching this and decided not > to, but only because it might confuse packagers if they see unrecognized > files in the installation. I realize now that c3a0818460a8 was > back-patched. Any opinions on backpatching? We've adde

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2020-10-15 Thread Alvaro Herrera
On 2020-Oct-15, Alvaro Herrera wrote: > Pushed, thanks. I forgot to mention that I considered backpatching this and decided not to, but only because it might confuse packagers if they see unrecognized files in the installation. I realize now that c3a0818460a8 was back-patched. Any opinions on b

Re: Aw: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Bruce Momjian
On Thu, Oct 15, 2020 at 12:01:21PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > We would want the timeline history file contents label changed from > > BYTEA to TEXT in the docs changed for all supported versions, add a C > > comment to all backbranches that BYTEA is the same as TEXT protoco

Re: Packaging - Packages names consistency (RPM)

2020-10-15 Thread Devrim Gündüz
Hi, RPM packager speaking: I agree that this is very annoying, and this is also in my todo list. Let me try to prioritize it. Regards, Devrim On 15 October 2020 17:23:33 GMT+03:00, Bruno Lavoie wrote: >Hi Hackers, > >First, thanks for working on such a great database! :) > >We're currently tr

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2020-10-15 Thread Alvaro Herrera
On 2020-Sep-30, Craig Ringer wrote: > On Tue, 29 Sep 2020 at 22:09, Alvaro Herrera > wrote: > > I happened to come across this thread by accident, and I tend to agree > > that we need to install both isolationtester and pg_isolation_regress to > > the pgxs dirs, just like we do pg_regress. I ca

Re: Aw: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Tom Lane
Bruce Momjian writes: > We would want the timeline history file contents label changed from > BYTEA to TEXT in the docs changed for all supported versions, add a C > comment to all backbranches that BYTEA is the same as TEXT protocol > fields, and change the C code to return TEXT IN PG 14. Is tha

Aw: Re: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Brar Piening
Tom Lane wrote: > Well, let's label it as text and then in the description > of the message, note that the field contains the raw file contents, > whose encoding is unknown to the server. +1 This would definitely have solved my initial issue and looks like a sane way forward without over-engin

Re: Aw: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Bruce Momjian
On Thu, Oct 15, 2020 at 11:18:33AM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Thu, Oct 15, 2020 at 10:03:46AM -0400, Tom Lane wrote: > >> I don't really see why our only documentation choices are "text" or > >> "bytea". Can't we just write "(raw file contents)" or the like? > > > Agre

Re: [Patch] Using Windows groups for SSPI authentication

2020-10-15 Thread Stephen Frost
Greetings, * Russell Foster (russell.foster.cod...@gmail.com) wrote: > Right after I sent that I realized that sspi-group was a bad idea, not sure > if that's even a thing. Tried to cancel as it was still in moderation, but > it made it through anyways! You are right, it is very windows specific.

Re: Aw: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Tom Lane
Bruce Momjian writes: > On Thu, Oct 15, 2020 at 10:03:46AM -0400, Tom Lane wrote: >> I don't really see why our only documentation choices are "text" or >> "bytea". Can't we just write "(raw file contents)" or the like? > Agreed. The reason they are those labels is that the C code has to > labe

Re: Logical Replication - detail message with names of missing columns

2020-10-15 Thread Alvaro Herrera
On 2020-Oct-06, Amit Kapila wrote: > I have made a few changes, (a) moved free of missingatts in the caller > where we are allocating it. (b) added/edited/removed comments, (c) ran > pgindent. This is committed as f07707099c17. Thanks

Re: Aw: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Bruce Momjian
On Thu, Oct 15, 2020 at 10:03:46AM -0400, Tom Lane wrote: > Brar Piening writes: > > Bruce Momjian wrote: > >> I did some more research on this. It turns out timeline 'content' is > >> the only BYTEA listed in the protocol docs, even though it just passes C > >> strings to pq_sendbytes(), just li

Re: partition routing layering in nodeModifyTable.c

2020-10-15 Thread Heikki Linnakangas
On Wed, Oct 14, 2020 at 6:04 PM Heikki Linnakangas wrote: I'll continue with the last couple of patches in this thread. I committed the move of the cross-partition logic to new ExecCrossPartitionUpdate() function, with just minor comment editing and pgindent. I left out the refactoring aroun

Re: Aw: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Fujii Masao
On 2020/10/15 23:03, Tom Lane wrote: Brar Piening writes: Bruce Momjian wrote: I did some more research on this. It turns out timeline 'content' is the only BYTEA listed in the protocol docs, even though it just passes C strings to pq_sendbytes(), just like many other fields like the field

Packaging - Packages names consistency (RPM)

2020-10-15 Thread Bruno Lavoie
Hi Hackers, First, thanks for working on such a great database! :) We're currently trying to automate our PostgreSQL setup by using Ansible. We have an Ansible role for which we can specify supplemental extensions for which a deployment must install. To keep it simple across deployed version we

Re: Is the doc for SHOW somewhat stale?

2020-10-15 Thread Fujii Masao
On 2020/10/06 16:53, Kyotaro Horiguchi wrote: Hello. If I read the documentation of the SHOW command, it says: https://www.postgresql.org/docs/13/sql-show.html name The name of a run-time parameter. Available parameters are documented in Chapter 19 and on the SET reference page. In a

Re: Aw: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Tom Lane
Brar Piening writes: > Bruce Momjian wrote: >> I did some more research on this. It turns out timeline 'content' is >> the only BYTEA listed in the protocol docs, even though it just passes C >> strings to pq_sendbytes(), just like many other fields like the field >> above it, the timeline histor

Re: Parallel Append can break run-time partition pruning

2020-10-15 Thread Amit Langote
Robert forwarded me a link to an email sent to -general list, where the reported problem seems to be the same problem that was being discussed here. https://www.postgresql.org/message-id/flat/d0f6d811-8946-eb9f-68e2-1a8a7f80ff21%40a-kretschmer.de Going over the last few emails, it seems that Davi

improve the algorithm cached_plan_cost with real planning time?

2020-10-15 Thread Andy Fan
In cached_plan_cost, we do consider the cost of planning, with the following algorithm. int nrelations = list_length(plannedstmt->rtable); result += 1000.0 * cpu_operator_cost * (nrelations + 1); I run into a case where 10 relations are joined, 3 of them have hundreds of partitions. at last n

Aw: Re: Minor documentation error regarding streaming replication protocol

2020-10-15 Thread Brar Piening
Bruce Momjian wrote: >> Good point. The reporter was assuming the data would come to the client >> in the bytea output format specified by the GUC, e.g. \x..., so that >> doesn't happen either. As I said before, it is more raw bytes, but we >> don't have an SQL data type for that. > I did some mo

Re: Wrong statistics for size of XLOG_SWITCH during pg_waldump.

2020-10-15 Thread movead...@highgo.ca
Thanks for all the suggestions. >Yeah. In its current shape, it means that only pg_waldump would be >able to know this information. If you make this information part of >xlogdesc.c, any consumer of the WAL record descriptions would be able >to show this information, so it would provide a consist

Re: pgsql: Restore replication protocol's duplicate command tags

2020-10-15 Thread Alvaro Herrera
On 2020-Oct-15, Amit Kapila wrote: > On Thu, Oct 15, 2020 at 6:07 AM Alvaro Herrera > wrote: > > On 2020-Oct-14, Alvaro Herrera wrote: > > Hm, this failed on sidewinder. > > Now, curculio [1] also seems to be failing for the same reason. Yeah ... and now they're both green. Anyway clearly th

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-15 Thread Amit Kapila
On Thu, Oct 15, 2020 at 9:56 AM Greg Nancarrow wrote: > > On Fri, Oct 9, 2020 at 8:09 PM Amit Kapila wrote: > > > > It might be a good idea to first just get this patch committed, if > > possible. So, I have reviewed the latest version of this patch: > > > > 0001-InsertParallelSelect > > I've att

Re: New statistics for tuning WAL buffer size

2020-10-15 Thread Fujii Masao
On 2020/10/13 11:57, Masahiro Ikeda wrote: On 2020-10-06 15:57, Masahiro Ikeda wrote: Hi, I think it's better to add other WAL statistics to the pg_stat_wal view. I'm thinking to add the following statistics. Please let me know your thoughts. 1.  Basic wal statistics * wal_records: Total n

Re: Parallel Inserts in CREATE TABLE AS

2020-10-15 Thread Amit Kapila
On Thu, Oct 15, 2020 at 9:14 AM Bharath Rupireddy wrote: > > On Wed, Oct 14, 2020 at 6:16 PM Amit Kapila wrote: > > > > > For prepared statements, the parallelism will not be picked and so is > > > parallel insertion. > > > > Hmm, I am not sure what makes you say this statement. The parallelism >

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-15 Thread Keisuke Kuroda
Hi Amit, Dilip, I am glad to solve this problem, Thanks! Best Regards, -- Keisuke Kuroda NTT Software Innovation Center keisuke.kuroda.3...@gmail.com

Re: Parallel copy

2020-10-15 Thread Amit Kapila
On Wed, Oct 14, 2020 at 6:51 PM vignesh C wrote: > > On Fri, Oct 9, 2020 at 11:01 AM Amit Kapila wrote: > > > > I am not able to properly parse the data but If understand the wal > > data for non-parallel (1116 | 0 | 3587203) and parallel (1119 > > | 6 | 3624405) case doesn't seem

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-15 Thread Amit Kapila
On Tue, Oct 13, 2020 at 5:41 AM Masahiko Sawada wrote: > > On Mon, 12 Oct 2020 at 23:45, Shinoda, Noriyoshi (PN Japan A&PS > Delivery) wrote: > > > > > > > As it may have been discussed, I think the 'name' column in > > > pg_stat_replication_slots is more consistent with the column name and > >

Re: Wrong statistics for size of XLOG_SWITCH during pg_waldump.

2020-10-15 Thread Kyotaro Horiguchi
At Thu, 15 Oct 2020 17:32:10 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 15 Oct 2020 12:56:02 +0800, "movead...@highgo.ca" > wrote in > > Thanks for all the suggestions. > > > > >Yeah. In its current shape, it means that only pg_waldump would be > > >able to know this information. If

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-10-15 Thread Andres Freund
Hi, On 2020-09-21 14:20:03 -0700, Andres Freund wrote: > I can give it a try. I can see several paths of varying invasiveness, > not sure yet what the best approach is. Let me think about if for a bit. Ugh, sorry for taking so long to get around to this. Attached is a *prototype* implemention of

Re: gs_group_1 crashing on 13beta2/s390x

2020-10-15 Thread Andres Freund
Hi, On 2020-10-14 17:56:16 -0700, Andres Freund wrote: > Oh dear. It's not as simple as that. The issue indeed are relocations, > but we don't hit those errors. The issue rather is that the systemz > specific relative redirection code thought that the only relative > symbols are functions. So it c

Re: Wrong statistics for size of XLOG_SWITCH during pg_waldump.

2020-10-15 Thread Kyotaro Horiguchi
At Thu, 15 Oct 2020 12:56:02 +0800, "movead...@highgo.ca" wrote in > Thanks for all the suggestions. > > >Yeah. In its current shape, it means that only pg_waldump would be > >able to know this information. If you make this information part of > >xlogdesc.c, any consumer of the WAL record des

Re: setLastTid() and currtid()

2020-10-15 Thread Michael Paquier
On Tue, Oct 13, 2020 at 02:12:53PM -0400, Bruce Momjian wrote: > Where are we on this? Can the functions be removed in PG 14? (Sent this message previously but it got lost after some cross-posting across two lists, issue fixed now.) I still have a patch lying around to do that, registered in the

Re: Online checksums verification in the backend

2020-10-15 Thread Julien Rouhaud
On Thu, Oct 15, 2020 at 1:37 PM Julien Rouhaud wrote: > > On Thu, Oct 1, 2020 at 1:07 PM Michael Paquier wrote: > > > > On Fri, Sep 25, 2020 at 06:11:47PM +0800, Julien Rouhaud wrote: > > > Thanks a lot for the tests! I'm not surprised that forcing the lock > > > will slow down the pg_check_rela

Re: Feature improvement for FETCH tab completion

2020-10-15 Thread Fujii Masao
On 2020/09/25 21:46, Fujii Masao wrote: On 2020/09/25 17:21, btnakamichin wrote: 2020-09-25 15:38 に Fujii Masao さんは書きました: On 2020/09/25 14:24, btnakamichin wrote: Hello! I’d like to improve the FETCH tab completion. The FETCH tab completion . Therefore, this patch fixes the problem. Pr

Re: Use appendStringInfoString and appendPQExpBufferStr where possible

2020-10-15 Thread David Rowley
On Wed, 14 Oct 2020 at 21:05, David Rowley wrote: > I've attached the patch I ended up with. I plan on pushing this in the > next few days. Pushed. David

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-15 Thread Dilip Kumar
On Thu, Oct 15, 2020 at 12:38 PM Amit Kapila wrote: > > On Wed, Oct 14, 2020 at 4:51 PM Dilip Kumar wrote: > > > > On Wed, Oct 14, 2020 at 4:12 PM Amit Kapila wrote: > > > > > > > > > Thanks for the tests. The latest patch looks mostly good to me. I have > > > made minor changes to the patch (a)

Possible memory leak in pgcrypto with EVP_MD_CTX

2020-10-15 Thread Michael Paquier
Hi all, It happens that pgcrypto has the following leak if a digest cannot be initialized: --- a/contrib/pgcrypto/openssl.c +++ b/contrib/pgcrypto/openssl.c @@ -202,6 +202,7 @@ px_find_digest(const char *name, PX_MD **res) } if (EVP_DigestInit_ex(ctx, md, NULL) == 0) { + EVP_MD_C

Re: Add Information during standby recovery conflicts

2020-10-15 Thread Masahiko Sawada
On Thu, 15 Oct 2020 at 14:52, Kyotaro Horiguchi wrote: > > At Thu, 15 Oct 2020 14:28:57 +0900, Masahiko Sawada > wrote in > > > ereport(..(errmsg("%s", _("hogehoge" results in > > > fprintf((translated("%s")), translate("hogehoge")). > > > > > > So your change (errmsg("%s", gettext_noop("hog

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-15 Thread Amit Kapila
On Wed, Oct 14, 2020 at 4:51 PM Dilip Kumar wrote: > > On Wed, Oct 14, 2020 at 4:12 PM Amit Kapila wrote: > > > > > > Thanks for the tests. The latest patch looks mostly good to me. I have > > made minor changes to the patch (a) changed the order where the new > > message is placed at one place t

Re: pgsql: Restore replication protocol's duplicate command tags

2020-10-15 Thread Amit Kapila
On Thu, Oct 15, 2020 at 6:07 AM Alvaro Herrera wrote: > > On 2020-Oct-14, Alvaro Herrera wrote: > > > Add a test case that shows the failure. It might still succeed even > > without the patch when run on a fast enough server, but it suffices to > > show the bug in enough cases that it would be no