RE: strange slow query - lost lot of time somewhere

2022-05-04 Thread Jakub Wartak
Hi Pavel, > I have not debug symbols, so I have not more details now > Breakpoint 1 at 0x7f557f0c16c0 > (gdb) c > Continuing. > Breakpoint 1, 0x7f557f0c16c0 in mmap64 () from /lib64/libc.so.6 > (gdb) bt > #0  0x7f557f0c16c0 in mmap64 () from /lib64/libc.so.6 > #1  0x7f557f04dd91 in sy

Re: Progress report removal of temp files and temp relation files using ereport_startup_progress

2022-05-04 Thread Bharath Rupireddy
On Mon, May 2, 2022 at 6:26 PM Ashutosh Bapat wrote: > > Hi Bharath, > > > On Sat, Apr 30, 2022 at 11:08 AM Bharath Rupireddy > wrote: > > > > Hi, > > > > At times, there can be many temp files (under pgsql_tmp) and temp > > relation files (under removal which after crash may take longer during >

Re: Perform streaming logical transactions by background workers and parallel apply

2022-05-04 Thread Peter Smith
Here are my review comments for v5-0001. I will take a look at the v5-0002 (TAP) patch another time. == 1. Commit message The message still refers to "apply background". Should that say "apply background worker"? Other parts just call this the "worker". Personally, I think it might be bett

Re: Skipping schema changes in publication

2022-05-04 Thread Amit Kapila
On Thu, May 5, 2022 at 9:20 AM Amit Kapila wrote: > > On Wed, May 4, 2022 at 7:05 PM Peter Eisentraut > wrote: > > > > On 14.04.22 15:47, Peter Eisentraut wrote: > > > That said, I'm not sure this feature is worth the trouble. If this is > > > useful, what about "whole database except these sche

Re: Proposal for internal Numeric to Uint64 conversion function.

2022-05-04 Thread Amul Sul
On Tue, May 3, 2022 at 8:04 PM Peter Eisentraut wrote: > > On 03.05.22 08:50, Amul Sul wrote: > >> Do you have any data that supports removing DirectionFunctionCall() > >> invocations? I suppose some performance benefit could be expected, or > >> what do you have in mind? > >> > > Not really, the

Re: Skipping schema changes in publication

2022-05-04 Thread Amit Kapila
On Wed, May 4, 2022 at 7:05 PM Peter Eisentraut wrote: > > On 14.04.22 15:47, Peter Eisentraut wrote: > > That said, I'm not sure this feature is worth the trouble. If this is > > useful, what about "whole database except these schemas"? What about > > "create this database from this template ex

Re: pg_stat_statements

2022-05-04 Thread Julien Rouhaud
Hi, On Tue, May 03, 2022 at 01:30:32PM +, Godfrin, Philippe E wrote: > > I wasn't exactly clear about the queries. The values clauses themselves are > not long - > We are using repeated values clauses: > > INSERT INTO timeseries.dvc_104 (tag_id, event_ts, bool_val, float_val, > int_val, stri

Re: Atomic GetFreeIndexPage()?

2022-05-04 Thread Peter Geoghegan
On Wed, May 4, 2022 at 12:16 PM Chris Cleveland wrote: > Similar code is repeated in a bunch of places. Each access method has to > explicitly write something into a freed page that identifies it as ok to use. I wouldn't say that that's what this code is doing, though I do see what you mean. The

Re: Costing elided SubqueryScans more nearly correctly

2022-05-04 Thread Tom Lane
I wrote: > I instrumented the code in setrefs.c, and found that during the > core regression tests this patch estimates correctly in 2103 > places while guessing wrongly in 54, so that seems like a pretty > good step forward. On second thought, that's not a terribly helpful summary. Breaking thin

Costing elided SubqueryScans more nearly correctly

2022-05-04 Thread Tom Lane
In [1] I complained about how SubqueryScans that get deleted from a plan tree by setrefs.c nonetheless contribute cost increments that might cause the planner to make odd choices. That turned out not to be the proximate cause of that particular issue, but it still seems like it might be a good ide

Re: SQL/JSON: FOR ORDINALITY bug

2022-05-04 Thread David G. Johnston
On Wed, May 4, 2022 at 1:43 PM David G. Johnston wrote: > On Wed, May 4, 2022 at 1:09 PM Erik Rijkers wrote: > >> Op 04-05-2022 om 21:12 schreef Andrew Dunstan: >> > >> >> I don't see how rowseq can be anything but 1. Each invocation of >> >> >> >> >> >> After some further experimenta

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-05-04 Thread David Zhang
Thanks a lot for the patch update. On 2022-05-02 1:25 a.m., Etsuro Fujita wrote: Hi, On Wed, Apr 20, 2022 at 4:55 AM David Zhang wrote: I tried to apply the patch to master and plan to run some tests, but got below errors due to other commits. I rebased the patch against HEAD. Attached is a

Re: SQL/JSON: FOR ORDINALITY bug

2022-05-04 Thread David G. Johnston
On Wed, May 4, 2022 at 1:09 PM Erik Rijkers wrote: > Op 04-05-2022 om 21:12 schreef Andrew Dunstan: > > > > I don't see how rowseq can be anything but 1. Each invocation of > >> > >> > >> After some further experimentation, I now think you must be right, > David. > >> > >> Also, lookin

Re: SQL/JSON: FOR ORDINALITY bug

2022-05-04 Thread Erik Rijkers
Op 04-05-2022 om 21:12 schreef Andrew Dunstan: I don't see how rowseq can be anything but 1.  Each invocation of After some further experimentation, I now think you must be right, David. Also, looking at the DB2 docs:   https://www.ibm.com/docs/en/i/7.2?topic=data-using-json-table     (s

Re: Did we intend to change whether PUBLIC can create tables in the public schema by default?

2022-05-04 Thread David G. Johnston
On Wed, May 4, 2022 at 12:42 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > Hey, > > For the following sequence of commands, on a newly initdb v15devel and > mostly clean v13 I get a failure and a created table respectively. > > Apparently I didn't search commit history well enough th

Did we intend to change whether PUBLIC can create tables in the public schema by default?

2022-05-04 Thread David G. Johnston
Hey, For the following sequence of commands, on a newly initdb v15devel and mostly clean v13 I get a failure and a created table respectively. Showing v15devel: postgres=# create database testdb; CREATE DATABASE postgres=# create role testrole; CREATE ROLE postgres=# \c testdb You are now connec

Re: Atomic GetFreeIndexPage()?

2022-05-04 Thread Tom Lane
Chris Cleveland writes: > Would it make sense to make GetFreeIndexPage() atomic? I strongly doubt it. The loss of concurrency would outweigh any code-simplicity benefits. regards, tom lane

Re: Query generates infinite loop

2022-05-04 Thread Tom Lane
Jeff Janes writes: > The regression test you added for this change causes an infinite loop when > run against an unpatched server with --install-check. That is a bit > unpleasant. Is there something we can and should do about that? I was > expecting regression test failures of course but not an

Atomic GetFreeIndexPage()?

2022-05-04 Thread Chris Cleveland
Would it make sense to make GetFreeIndexPage() atomic? Internally, GetFreeIndexPage() calls GetPageWithFreeSpace() and then RecordUsedIndexPage() in two separate operations. It's possible for two different processes to get the same free page at the same time. To guard against this, there are seve

Re: JSON Functions and Operators Docs for v15

2022-05-04 Thread Andrew Dunstan
On 2022-05-04 We 11:39, Tom Lane wrote: > "David G. Johnston" writes: >> Is there a thread I'm not finding where the upcoming JSON function >> documentation is being made reasonably usable after doubling its size with >> all the new JSON Table features that we've added? If nothing else, the >>

Re: SQL/JSON: FOR ORDINALITY bug

2022-05-04 Thread Andrew Dunstan
On 2022-05-04 We 10:39, Erik Rijkers wrote: > Op 04-05-2022 om 13:55 schreef Andrew Dunstan: >> >> On 2022-05-03 Tu 20:39, David G. Johnston wrote: >>> On Tue, May 3, 2022 at 5:27 PM Andrew Dunstan >>> wrote: >>> >>> >>> On 2022-05-03 Tu 11:19, Erik Rijkers wrote: >>> > Hi >>> > >

Re: Query generates infinite loop

2022-05-04 Thread Jeff Janes
On Wed, Apr 20, 2022 at 5:43 PM Tom Lane wrote: > I wrote: > > it's true that infinities as generate_series endpoints are going > > to work pretty oddly, so I agree with the idea of forbidding 'em. > > > Numeric has infinity as of late, so the numeric variant would > > need to do this too. > > Oh

Re: fix cost subqueryscan wrong parallel cost

2022-05-04 Thread Tom Lane
Robert Haas writes: > On Tue, May 3, 2022 at 2:13 PM Tom Lane wrote: >> In any case, fundamental redesign of what EXPLAIN prints is a job >> for v16 or later. Are you okay with the proposed patch as a v15 fix? > Yes. I can't really vouch for it, but I don't object to it. I re-read the patch an

Re: strange slow query - lost lot of time somewhere

2022-05-04 Thread Pavel Stehule
st 4. 5. 2022 v 16:08 odesílatel Jakub Wartak napsal: > > Additional three ways to figure that one (all are IMHO production safe): > a) already mentioned perf with --call-graph dwarf -p PID > b) strace -p PID -e 'mmap' # verify if mmap() NULL is not having > MAP_ANONYMOUS flag, size of mmap() req

Re: strange slow query - lost lot of time somewhere

2022-05-04 Thread Pavel Stehule
st 4. 5. 2022 v 2:15 odesílatel David Rowley napsal: > On Tue, 3 May 2022 at 17:02, Pavel Stehule > wrote: > > út 3. 5. 2022 v 6:57 odesílatel Tom Lane napsal: > >> You sure there's not something taking an exclusive lock on one of these > >> tables every so often? > > > > I am almost sure, I ca

TAP test fail: we don't always detect backend crashes

2022-05-04 Thread Tom Lane
I discovered while poking at an LDAP problem that our TAP tests are 100% reproducibly capable of ignoring server crashes and reporting success anyway. This problem occurs if the postmaster doesn't get the child failure report until after shutdown has been initiated, in which case you find somethin

Re: automatically generating node support functions

2022-05-04 Thread Alvaro Herrera
On 2022-May-04, Peter Eisentraut wrote: > I have committed your change to the JsonTableColumnType enum and the removal > of JsonPathSpec. Thanks! > Other than that and some whitespace changes, I didn't find anything in > your 0002 patch that was different from my last submitted patch. Did > I m

Re: automatically generating node support functions

2022-05-04 Thread Peter Eisentraut
On 19.04.22 13:40, Alvaro Herrera wrote: I rebased this mostly out of curiousity. I fixed some smallish conflicts and fixed a typedef problem new in JSON support; however, even with these fixes it doesn't compile, because JsonPathSpec uses a novel typedef pattern that apparently will need bespok

Re: JSON Functions and Operators Docs for v15

2022-05-04 Thread David G. Johnston
On Wed, May 4, 2022 at 8:39 AM Tom Lane wrote: > "David G. Johnston" writes: > > Is there a thread I'm not finding where the upcoming JSON function > > documentation is being made reasonably usable after doubling its size > with > > all the new JSON Table features that we've added? If nothing e

Re: JSON Functions and Operators Docs for v15

2022-05-04 Thread Justin Pryzby
On Wed, May 04, 2022 at 08:32:51AM -0700, David G. Johnston wrote: > Hey, > > Is there a thread I'm not finding where the upcoming JSON function > documentation is being made reasonably usable after doubling its size with > all the new JSON Table features that we've added? If nothing else, the >

Re: JSON Functions and Operators Docs for v15

2022-05-04 Thread Tom Lane
"David G. Johnston" writes: > Is there a thread I'm not finding where the upcoming JSON function > documentation is being made reasonably usable after doubling its size with > all the new JSON Table features that we've added? If nothing else, the > table of contents at the top of the page needs t

JSON Functions and Operators Docs for v15

2022-05-04 Thread David G. Johnston
Hey, Is there a thread I'm not finding where the upcoming JSON function documentation is being made reasonably usable after doubling its size with all the new JSON Table features that we've added? If nothing else, the table of contents at the top of the page needs to be greatly expanded to make s

Re: [PATCH] Completed unaccent dictionary with many missing characters

2022-05-04 Thread Tom Lane
Peter Eisentraut writes: > On 28.04.22 18:50, Przemysław Sztoch wrote: >> Current unnaccent dictionary does not include many popular numeric symbols, >> in example: "m²" -> "m2" > Seems reasonable. It kinda feels like this is outside the charter of an "unaccent" dictionary. I don't object to ha

Re: configure openldap crash warning

2022-05-04 Thread Tom Lane
I wrote: > Peter Eisentraut writes: >> I tried building with Homebrew-supplied openldap. What ends up >> happening is that the postgres binary is indeed linked with openldap, >> but libpq still is linked against the OS-supplied LDAP framework. >> (Checked with "otool -L" in each case.) Can so

Re: [PATCH] Completed unaccent dictionary with many missing characters

2022-05-04 Thread Peter Eisentraut
On 28.04.22 18:50, Przemysław Sztoch wrote: Current unnaccent dictionary does not include many popular numeric symbols, in example: "m²" -> "m2" Seems reasonable. Can you explain what your patch does to achieve this?

Re: configure openldap crash warning

2022-05-04 Thread Tom Lane
Peter Eisentraut writes: > On 02.05.22 16:03, Tom Lane wrote: >> I'm not that excited about getting rid of this warning, because to the >> extent that anyone notices it at all, it'll motivate them to get OpenLDAP >> from Homebrew or MacPorts, which seems like a good thing. > I tried building with

Re: SQL/JSON: FOR ORDINALITY bug

2022-05-04 Thread Erik Rijkers
Op 04-05-2022 om 13:55 schreef Andrew Dunstan: On 2022-05-03 Tu 20:39, David G. Johnston wrote: On Tue, May 3, 2022 at 5:27 PM Andrew Dunstan wrote: On 2022-05-03 Tu 11:19, Erik Rijkers wrote: > Hi > > I've copied some statements from the .pdf called: > "TECHNICAL RE

RE: strange slow query - lost lot of time somewhere

2022-05-04 Thread Jakub Wartak
> I do agree that the perf report does indicate that the extra time is taken > due to > some large amount of memory being allocated. I just can't quite see how that > would happen in Memoize given that > estimate_num_groups() clamps the distinct estimate as the number of input > rows, which is 91

Re: configure openldap crash warning

2022-05-04 Thread Peter Eisentraut
On 02.05.22 16:03, Tom Lane wrote: I'm not that excited about getting rid of this warning, because to the extent that anyone notices it at all, it'll motivate them to get OpenLDAP from Homebrew or MacPorts, which seems like a good thing. I tried building with Homebrew-supplied openldap. What e

Re: Add a new function and a document page to get/show all the server hooks

2022-05-04 Thread Tom Lane
Peter Eisentraut writes: > On 04.05.22 12:54, Bharath Rupireddy wrote: >> One problem is that the new function and doc page create an extra >> burden of keeping them up to date with the hooks modifications and new >> hook additions, but I think that can be taken care of in the review >> phases. >

Re: bogus: logical replication rows/cols combinations

2022-05-04 Thread Peter Eisentraut
On 03.05.22 21:40, Tomas Vondra wrote: So what's wrong with merging the column lists as implemented in the v2 patch, posted a couple days ago? Merging the column lists is ok if all other publication attributes match. Otherwise, I think not. I don't think triggers are a suitable alternati

Re: [PATCH] Log details for client certificate failures

2022-05-04 Thread Peter Eisentraut
On 04.05.22 01:05, Jacob Champion wrote: On Tue, 2022-05-03 at 21:06 +0200, Peter Eisentraut wrote: The information in pg_stat_ssl is limited to NAMEDATALEN (see struct PgBackendSSLStatus). It might make sense to align what your patch prints to identify certificates with what is shown in that

Re: Add pg_strtoupper and pg_strtolower functions

2022-05-04 Thread Tom Lane
Alvaro Herrera writes: > Currently, pg_toupper/pg_tolower are used in very limited situations. > Are they really always safe enough to run in arbitrary situations, > enough to create this new layer on top of them? They are not, and we should absolutely not be encouraging additional uses of them.

Re: Add a new function and a document page to get/show all the server hooks

2022-05-04 Thread Peter Eisentraut
On 04.05.22 12:54, Bharath Rupireddy wrote: One problem is that the new function and doc page create an extra burden of keeping them up to date with the hooks modifications and new hook additions, but I think that can be taken care of in the review phases. Thoughts? I think this has been propo

Re: Skipping schema changes in publication

2022-05-04 Thread Peter Eisentraut
On 14.04.22 15:47, Peter Eisentraut wrote: That said, I'm not sure this feature is worth the trouble.  If this is useful, what about "whole database except these schemas"?  What about "create this database from this template except these schemas".  This could get out of hand.  I think we should

Re: Add pg_strtoupper and pg_strtolower functions

2022-05-04 Thread Alvaro Herrera
On 2022-May-02, Bharath Rupireddy wrote: > Hi, > > I came across pg_toupper and pg_tolower functions, converting a single > character, are being used in loops to convert an entire > null-terminated string. The cost of calling these character-based > conversion functions (even though small) can be

Re: testclient.exe installed under MSVC

2022-05-04 Thread Daniel Gustafsson
> On 4 May 2022, at 02:34, Andrew Dunstan wrote: > I think we should make the standard MSVC install look as much like the > standard Unix/msys install as possible. +1 -- Daniel Gustafsson https://vmware.com/

Re: testclient.exe installed under MSVC

2022-05-04 Thread Daniel Gustafsson
> On 3 May 2022, at 16:50, Daniel Gustafsson wrote: > >> On 3 May 2022, at 15:58, Alvaro Herrera wrote: >> >> On 2022-May-03, Noah Misch wrote: >> >>> Michael Paquier recommended s/-/_/ for uri-regress, and I agree with that. >>> What do you think? >> >> libpq_uri-regress is horrible, so +1 f

Re: SQL/JSON: FOR ORDINALITY bug

2022-05-04 Thread Andrew Dunstan
On 2022-05-03 Tu 20:39, David G. Johnston wrote: > On Tue, May 3, 2022 at 5:27 PM Andrew Dunstan wrote: > > > On 2022-05-03 Tu 11:19, Erik Rijkers wrote: > > Hi > > > > I've copied some statements from the .pdf called: > > "TECHNICAL REPORT ISO/IEC TR 19075-6   First edition

Re: [RFC] building postgres with meson -v8

2022-05-04 Thread Peter Eisentraut
More patches: 0001-meson-Assorted-compiler-test-tweaks.patch I was going through a diff of pg_config.h between old and new build and found a few omissions and small differences. Some of the blah ? 1 : false is of course annoying and can be removed eventually, but it's useful when analy

Re: Configuration Parameter/GUC value validation hook

2022-05-04 Thread Bharath Rupireddy
On Tue, May 3, 2022 at 10:43 PM Robert Haas wrote: > > On Tue, May 3, 2022 at 11:45 AM Tom Lane wrote: > > Robert Haas writes: > > > I have some desire here to see us solve this problem not just for > > > service providers, but for users in general. You don't have to be a > > > service provider

Re: Add pg_strtoupper and pg_strtolower functions

2022-05-04 Thread Bharath Rupireddy
On Mon, May 2, 2022 at 6:43 PM Ashutosh Bapat wrote: > > On Mon, May 2, 2022 at 6:21 PM Bharath Rupireddy > wrote: > > > > Hi, > > > > I came across pg_toupper and pg_tolower functions, converting a single > > character, are being used in loops to convert an entire > > null-terminated string. The

Add a new function and a document page to get/show all the server hooks

2022-05-04 Thread Bharath Rupireddy
Hi, As we have many hooks in postgres that extends the postgres functionality, I'm wondering if it's a good idea to add a new function, say, pg_get_all_server_hooks, returning hook name, hook declaration and its current value (if there's any external module loaded implementing the hook). This basi

Re: Hash index build performance tweak from sorting

2022-05-04 Thread Amit Kapila
On Mon, May 2, 2022 at 9:28 PM Simon Riggs wrote: > > On Sat, 30 Apr 2022 at 12:12, Amit Kapila wrote: > > > > On Tue, Apr 19, 2022 at 3:05 AM Simon Riggs > > wrote: > > > > > > Hash index pages are stored in sorted order, but we don't prepare the > > > data correctly. > > > > > > We sort the da

Re: Logical replication timeout problem

2022-05-04 Thread Amit Kapila
On Mon, May 2, 2022 at 8:07 AM Masahiko Sawada wrote: > > On Mon, May 2, 2022 at 11:32 AM Amit Kapila wrote: > > > > > > So, shall we go back to the previous approach of using a separate > > function update_replication_progress? > > Ok, agreed. > Attached, please find the updated patch according