Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-09-26 Thread Dilip Kumar
On Thu, Sep 2, 2021 at 8:52 PM Robert Haas wrote: PFA, updated version of the patch, where I have fixed the issues reported by you and also done some more refactoring and patch split, next I am planning to post the patch with another approach where we scan the directory instead of scanning the pg

RE: Added schema level support for publication.

2021-09-26 Thread houzj.f...@fujitsu.com
On Monday, September 27, 2021 12:32 PM vignesh C wrote: > On Fri, Sep 24, 2021 at 11:46 AM vignesh C wrote: > > > > Attached v33 patch has the preprocess_pubobj_list review comment fix > suggested by Alvaro at [1]. The > v33-0006-Alternate-grammar-for-ALL-TABLES-IN-SCHEMA.patch patch has > the g

Re: Timeout failure in 019_replslot_limit.pl

2021-09-26 Thread Michael Paquier
On Mon, Sep 27, 2021 at 11:53:07AM +0530, Amit Kapila wrote: > So, it seems on your machine it has passed the following condition in > secure_write: > if (n < 0 && !port->noblock && (errno == EWOULDBLOCK || errno == EAGAIN)) Yep. > If so, this indicates write failure which seems odd to me and pr

Re: Timeout failure in 019_replslot_limit.pl

2021-09-26 Thread Amit Kapila
On Mon, Sep 27, 2021 at 11:32 AM Michael Paquier wrote: > > On Sat, Sep 25, 2021 at 05:12:42PM +0530, Amit Kapila wrote: > > Now, in the failed run, it appears that due to some reason WAL sender > > has not released the slot. Is it possible to see if the WAL sender is > > still alive when a checkp

Re: Timeout failure in 019_replslot_limit.pl

2021-09-26 Thread Michael Paquier
On Sat, Sep 25, 2021 at 05:12:42PM +0530, Amit Kapila wrote: > Now, in the failed run, it appears that due to some reason WAL sender > has not released the slot. Is it possible to see if the WAL sender is > still alive when a checkpoint is stuck at ConditionVariableSleep? And > if it is active, wha

Re: Skipping logical replication transactions on subscriber side

2021-09-26 Thread Amit Kapila
On Mon, Sep 27, 2021 at 11:02 AM Masahiko Sawada wrote: > > On Mon, Sep 27, 2021 at 12:50 PM Masahiko Sawada > wrote: > > > > On Mon, Sep 27, 2021 at 12:24 PM Amit Kapila > > wrote: > > > > > > On Mon, Sep 27, 2021 at 6:21 AM Masahiko Sawada > > > wrote: > > > > > > > > On Sat, Sep 25, 2021

Re: Skipping logical replication transactions on subscriber side

2021-09-26 Thread Amit Kapila
On Mon, Sep 27, 2021 at 11:20 AM Masahiko Sawada wrote: > > On Mon, Sep 27, 2021 at 12:46 PM Amit Kapila wrote: > > > > On Fri, Sep 24, 2021 at 7:01 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Sep 3, 2021 at 4:33 AM Mark Dilger > > > wrote: > > > > > > > I am attaching a version of such

Re: Next Steps with Hash Indexes

2021-09-26 Thread Amit Kapila
On Thu, Sep 23, 2021 at 11:11 AM Dilip Kumar wrote: > > On Thu, Sep 23, 2021 at 10:04 AM Sadhuprasad Patro wrote: > > > > And to get the multi-column hash index selected, we may set > > enable_hashjoin =off, to avoid any condition become join condition, > > saw similar behaviors in other DBs as w

Re: Skipping logical replication transactions on subscriber side

2021-09-26 Thread Masahiko Sawada
On Mon, Sep 27, 2021 at 12:46 PM Amit Kapila wrote: > > On Fri, Sep 24, 2021 at 7:01 AM Masahiko Sawada wrote: > > > > On Fri, Sep 3, 2021 at 4:33 AM Mark Dilger > > wrote: > > > > > I am attaching a version of such a function, plus some tests of your > > > patch (since it does not appear to h

Re: Skipping logical replication transactions on subscriber side

2021-09-26 Thread Masahiko Sawada
On Mon, Sep 27, 2021 at 12:50 PM Masahiko Sawada wrote: > > On Mon, Sep 27, 2021 at 12:24 PM Amit Kapila wrote: > > > > On Mon, Sep 27, 2021 at 6:21 AM Masahiko Sawada > > wrote: > > > > > > On Sat, Sep 25, 2021 at 4:23 PM Amit Kapila > > > wrote: > > > > > > > > Sure, but each tablesync work

Re: typos (and more)

2021-09-26 Thread Michael Paquier
On Mon, Sep 27, 2021 at 09:24:27AM +0900, Michael Paquier wrote: > Yes, thanks for compiling all these. The two changes committed were > the only user-visible changes, which is why I have hastened this part > to include those fixes. The rest could just go on HEAD. I have looked at the full set,

Re: Gather performance analysis

2021-09-26 Thread Dilip Kumar
On Sun, Sep 26, 2021 at 11:21 AM Dilip Kumar wrote: > > On Sat, Sep 25, 2021 at 2:18 AM Tomas Vondra > wrote: > > > > On 9/24/21 7:08 PM, Robert Haas wrote: > > > On Fri, Sep 24, 2021 at 3:50 AM Dilip Kumar wrote: > > >> Tomas, can you share your test script, I would like to repeat the same > >

Re: Failed transaction statistics to measure the logical replication progress

2021-09-26 Thread Amit Kapila
On Wed, Sep 22, 2021 at 10:10 AM osumi.takami...@fujitsu.com wrote: > > Just conducted some cosmetic changes > and rebased my patch, using v14 patch-set in [1]. > IIUC, this proposal will allow new xact stats for subscriptions via pg_stat_subscription. One thing that is not clear to me in this pa

Re: Remove page-read callback from XLogReaderState.

2021-09-26 Thread Thomas Munro
On Thu, Jul 15, 2021 at 4:48 PM Kyotaro Horiguchi wrote: > Gah... Thank you for noticing me. I thought that I have sent the > rebased version. This is the rebased version on the current master. Hi Kyotaro, Did you see this? https://www.postgresql.org/message-id/20210429022553.4h5qii5jb5eclu4i%

RE: Allow escape in application_name

2021-09-26 Thread kuroda.hay...@fujitsu.com
Dear Fujii-san, Horiguchi-san I'm sorry for sending a bad patch... > --- > elog.c:2785:14: warning: expression which evaluates to zero treated as a null > pointer constant of type 'char *' [-Wnon-literal-null-conversion] > *endptr = '\0'; >

Re: row filtering for logical replication

2021-09-26 Thread Amit Kapila
On Sat, Sep 25, 2021 at 3:36 PM Tomas Vondra wrote: > > On 9/25/21 6:23 AM, Amit Kapila wrote: > > On Sat, Sep 25, 2021 at 3:30 AM Tomas Vondra > > wrote: > >> > >> On 9/24/21 7:20 AM, Amit Kapila wrote: > >>> > >>> I think the right way to support functions is by the explicit marking > >>> of fu

Re: extensible options syntax for replication parser?

2021-09-26 Thread Ajin Cherian
On Mon, Sep 27, 2021 at 11:20 AM Ajin Cherian wrote: > > On Sat, Sep 25, 2021 at 4:28 AM tushar wrote: > > > > On 9/24/21 10:36 PM, Robert Haas wrote: > > > Here's v9, fixing the issue reported by Fujii Masao. > > > > Please refer this scenario where publication on v14RC1 and subscription > > on

Re: Skipping logical replication transactions on subscriber side

2021-09-26 Thread Masahiko Sawada
On Mon, Sep 27, 2021 at 12:24 PM Amit Kapila wrote: > > On Mon, Sep 27, 2021 at 6:21 AM Masahiko Sawada wrote: > > > > On Sat, Sep 25, 2021 at 4:23 PM Amit Kapila wrote: > > > > > > Sure, but each tablesync worker must have a separate relid. Why can't > > > we have a single hash table for both a

Re: Skipping logical replication transactions on subscriber side

2021-09-26 Thread Amit Kapila
On Fri, Sep 24, 2021 at 7:01 AM Masahiko Sawada wrote: > > On Fri, Sep 3, 2021 at 4:33 AM Mark Dilger > wrote: > > > I am attaching a version of such a function, plus some tests of your patch > > (since it does not appear to have any). Would you mind reviewing these and > > giving comments or

Re: Use simplehash.h instead of dynahash in SMgr

2021-09-26 Thread David Rowley
On Fri, 24 Sept 2021 at 20:26, Jaime Casanova wrote: > Are you planning to work on this in this CF? > This is marked as "Ready for committer" but it doesn't apply anymore. I've attached an updated patch. Since this patch is pretty different from the one that was marked as ready for committer, I'l

Re: Skipping logical replication transactions on subscriber side

2021-09-26 Thread Amit Kapila
On Mon, Sep 27, 2021 at 6:21 AM Masahiko Sawada wrote: > > On Sat, Sep 25, 2021 at 4:23 PM Amit Kapila wrote: > > > > Sure, but each tablesync worker must have a separate relid. Why can't > > we have a single hash table for both apply and table sync workers > > which are hashed by sub_id + rel_id

Re: decoupling table and index vacuum

2021-09-26 Thread Masahiko Sawada
On Sat, Sep 25, 2021 at 10:17 AM Peter Geoghegan wrote: > > On Thu, Sep 23, 2021 at 10:42 PM Masahiko Sawada > wrote: > > On Thu, Sep 16, 2021 at 7:09 AM Peter Geoghegan wrote: > > > Enabling index-only scans is a good enough reason to pursue this > > > project, even on its own. > > > > +1 > >

Re: Improve logging when using Huge Pages

2021-09-26 Thread Kyotaro Horiguchi
At Tue, 21 Sep 2021 19:23:22 -0500, Justin Pryzby wrote in > On Wed, Sep 22, 2021 at 02:03:11AM +0900, Fujii Masao wrote: > > Another idea is to output "Anonymous shared memory was allocated with > > huge pages" when it's successfully allocated with huge pages, and to output > > "Anonymous sha

Re: When is int32 not an int32?

2021-09-26 Thread David E. Wheeler
On Sep 26, 2021, at 19:25, Tom Lane wrote: > More to the point, you should be checking whether strtol reports overflow. > Having now seen your code, I'll opine that the failing platforms have > 32-bit long. Thanks for the pointer, Tom. I believe this fixes that particular issue. https://githu

PostgreSQL High Precision Mathematics Extension.

2021-09-26 Thread A Z
I have been trying to find active interest in a free for all use PostgreSQL extension, complete and available, on the public internet, that will support the following: # High Precision Numeric and Elementary Functions Support # #

Re: extensible options syntax for replication parser?

2021-09-26 Thread Ajin Cherian
On Sat, Sep 25, 2021 at 4:28 AM tushar wrote: > > On 9/24/21 10:36 PM, Robert Haas wrote: > > Here's v9, fixing the issue reported by Fujii Masao. > > Please refer this scenario where publication on v14RC1 and subscription > on HEAD (w/patch) > > --create a subscription with parameter two_phase=1

Re: PG 14 release notes, first draft

2021-09-26 Thread Yugo NAGATA
On Mon, 10 May 2021 02:03:08 -0400 Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still need

Re: Evaluate expression at planning time for two more cases

2021-09-26 Thread Jaime Casanova
On Tue, Jul 06, 2021 at 01:09:56PM +1200, David Rowley wrote: > On Tue, 9 Mar 2021 at 05:13, Ibrar Ahmed wrote: > > It was a minor change therefore I rebased the patch, please take a look. > [...] > > This is not really an acceptable way to determine the notnull > attribute value. Andy Fan propo

Re: Skipping logical replication transactions on subscriber side

2021-09-26 Thread Masahiko Sawada
On Sat, Sep 25, 2021 at 4:23 PM Amit Kapila wrote: > > On Fri, Sep 24, 2021 at 6:44 PM Masahiko Sawada wrote: > > > > On Fri, Sep 24, 2021 at 8:01 PM Amit Kapila wrote: > > > > > > > > 6. > > > +typedef struct PgStat_StatSubEntry > > > +{ > > > + Oid subid; /* hash table key */ > > > + > > > + /

Re: can we add some file(msvc) to gitignore

2021-09-26 Thread Michael Paquier
On Sun, Sep 26, 2021 at 12:18:01PM +0200, Daniel Gustafsson wrote: > As postgres isn't mandating a specific IDE or dev environment, we typically > don't add these files to the .gitignore we ship. If we did it would be an > enormous list we'd have to curate and maintain. Instead, everyone hacking

Re: typos (and more)

2021-09-26 Thread Michael Paquier
On Sun, Sep 26, 2021 at 12:01:17PM -0500, Justin Pryzby wrote: > Thanks Michael for applying fixes to user-facing docs (I hadn't realized that > the 2nd one needed to be backpatched). Yes, thanks for compiling all these. The two changes committed were the only user-visible changes, which is why I

Add prefix pg_catalog to pg_get_statisticsobjdef_columns() in describe.c (\dX)

2021-09-26 Thread Tatsuro Yamada
Hi, I found other functions that we should add "pg_catalog" prefix in describe.c. This fix is similar to the following commit. = commit 359bcf775550aa577c86ea30a6d071487fcca1ed Author: Alvaro Herrera Date: Sat Aug 28 12:04:15 2021 -0400 psql \dX: reference regclass with "pg_catalog."

Re: should INSERT SELECT use a BulkInsertState?

2021-09-26 Thread Justin Pryzby
On Mon, May 11, 2020 at 03:19:34PM +0900, Michael Paquier wrote: > On Fri, May 08, 2020 at 02:25:45AM -0500, Justin Pryzby wrote: > > Seems to me it should, at least conditionally. At least if there's a > > function > > scan or a relation or .. > > > > I mentioned a bit about our use-case here: >

Re: When is int32 not an int32?

2021-09-26 Thread Tom Lane
"David E. Wheeler" writes: > On Sep 26, 2021, at 18:31, Tom Lane wrote: >> I'd bet more along the lines of "your overflow check is less portable than >> you thought”. > Oh well now that you mention it and I look past things, I see we’re using > INT_MAX, but should probably use INT32_MAX. More

Re: When is int32 not an int32?

2021-09-26 Thread David E. Wheeler
On Sep 26, 2021, at 18:31, Tom Lane wrote: > I'd bet more along the lines of "your overflow check is less portable than > you thought”. Oh well now that you mention it and I look past things, I see we’re using INT_MAX, but should probably use INT32_MAX. https://github.com/theory/pg-semver/b

Re: Add create and update timestamp to all objects

2021-09-26 Thread Chapman Flack
On 09/26/21 18:28, Tom Lane wrote: > "Efrain J. Berdecia" writes: >> Are there any plans to add a create and last updated time stamp field to any >> and all objects in postgres? > > No. That said, if you'd be satisfied with a create OR last updated time, there should already be a txid there. I

Re: When is int32 not an int32?

2021-09-26 Thread David Fetter
On Sun, Sep 26, 2021 at 05:32:11PM -0400, David E. Wheeler wrote: > Hell Hackers, long time no email! > > I got a bug report for the semver extension: > > https://github.com/theory/pg-semver/issues/58 > > It claims that a test unexpected passes. That is, Test #31 is expected to > fail, becaus

Re: When is int32 not an int32?

2021-09-26 Thread Tom Lane
"David E. Wheeler" writes: > It claims that a test unexpected passes. That is, Test #31 is expected to > fail, because it intentionally tests a version in which its parts overflow > the int32[3] they’re stored in, with the expectation that one day we can > refactor the type to handle larger ver

Re: Add create and update timestamp to all objects

2021-09-26 Thread Tom Lane
"Efrain J. Berdecia" writes: > Are there any plans to add a create and last updated time stamp field to any > and all objects in postgres? No. This has been proposed and rejected (more than once). regards, tom lane

Re: Spelling change in LLVM 14 API

2021-09-26 Thread Thomas Munro
On Fri, Sep 24, 2021 at 12:52 PM Thomas Munro wrote: > Yeah, that's already stopped working since you wrote it a few weeks > ago... There's also been a second breakage site in our code due to > LLVM commit 85b732b5. New fix attached. Tested on LLVM 7, 9, 13, 14 > (= LLVM main branch commit 945df

When is int32 not an int32?

2021-09-26 Thread David E. Wheeler
Hell Hackers, long time no email! I got a bug report for the semver extension: https://github.com/theory/pg-semver/issues/58 It claims that a test unexpected passes. That is, Test #31 is expected to fail, because it intentionally tests a version in which its parts overflow the int32[3] they’

Re: Empty string in lexeme for tsvector

2021-09-26 Thread Jean-Christophe Arnu
Le dim. 26 sept. 2021 à 15:55, Artur Zakirov a écrit : > Nice catch! The patch looks good to me. > Can you also add a more general test case: > > =# SELECT $$'' '1' '2'$$::tsvector; > ERROR: syntax error in tsvector: "'' '1' '2'" > LINE 1: SELECT $$'' '1' '2'$$::tsvector; > > Thank you, Artur fo

Re: Add create and update timestamp to all objects

2021-09-26 Thread Nikolay Samokhvalov
On Sun, Sep 26, 2021 at 1:11 PM Efrain J. Berdecia wrote: > Are there any plans to add a create and last updated time stamp field to > any and all objects in postgres? > > Possibly even adding a updated_by documenting which role created and last > updated the object. > > All done natively and wit

Re: extended stats on partitioned tables

2021-09-26 Thread Justin Pryzby
On Sat, Sep 25, 2021 at 05:31:52PM -0500, Justin Pryzby wrote: > It seems like your patch should also check "inh" in examine_variable and > statext_expressions_load. I tried adding that - I mostly kept my patches separate. Hopefully this is more helpful than a complication. I added at: https://com

Add create and update timestamp to all objects

2021-09-26 Thread Efrain J. Berdecia
Are there any plans to add a create and last updated time stamp field to any and all objects in postgres? Possibly even adding a updated_by documenting which role created and last updated the object. All done natively and without the need for extra extensions. Thanks in advance.

Re: Release SPI plans for referential integrity with DISCARD ALL

2021-09-26 Thread Tom Lane
yuzuko writes: > I improved a patch according to Peter's following comment : >> but I think the >> solution of dropping all cached plans as part of DISCARD ALL seems a bit >> too extreme of a solution. In the context of connection pooling, >> getting a new session with pre-cached plans seems lik

Re: Add missing function abs (interval)

2021-09-26 Thread Isaac Morland
On Sun, 26 Sept 2021 at 13:42, Tom Lane wrote: > I wrote: > > Isaac Morland writes: > >> I've attached a patch for this. Turns out there was a comment in the > source > >> explaining that there is no interval_abs because it's not clear what to > >> return; but I think it's clear that if i is an

Re: Add missing function abs (interval)

2021-09-26 Thread Tom Lane
I wrote: > Isaac Morland writes: >> I've attached a patch for this. Turns out there was a comment in the source >> explaining that there is no interval_abs because it's not clear what to >> return; but I think it's clear that if i is an interval the larger of i and >> -i should be considered to be

Useless division(s) in interval_cmp_value

2021-09-26 Thread Tom Lane
While thinking about Isaac Morland's patch to add abs(interval), I happened to notice that interval_cmp_value() seems rather inefficently written: it's expending an int64 division -- or even two of them, if the compiler's not very smart -- to split up the "time" field into days and microseconds. T

Re: typos (and more)

2021-09-26 Thread Justin Pryzby
On Fri, Sep 24, 2021 at 04:58:27PM -0500, Justin Pryzby wrote: > A compilation of fixes for master. Thanks Michael for applying fixes to user-facing docs (I hadn't realized that the 2nd one needed to be backpatched). This fixes an file I failed to include in the "recheck" patch and more typos for

Re: Empty string in lexeme for tsvector

2021-09-26 Thread Artur Zakirov
On Fri, Sep 24, 2021 at 2:39 PM Jean-Christophe Arnu wrote: > Here's a new patch file taking your comments into account. Nice catch! The patch looks good to me. Can you also add a more general test case: =# SELECT $$'' '1' '2'$$::tsvector; ERROR: syntax error in tsvector: "'' '1' '2'" LINE 1: S

Re: extended stats on partitioned tables

2021-09-26 Thread Tomas Vondra
On 9/25/21 11:46 PM, Justin Pryzby wrote: > On Sat, Sep 25, 2021 at 11:01:21PM +0200, Tomas Vondra wrote: >>> Do you think it's possible to backpatch a fix to handle partitioned tables >>> specifically ? >>> >>> The "tuple already updated" error which I reported and which was fixed by >>> 859b3003

Re: can we add some file(msvc) to gitignore

2021-09-26 Thread Daniel Gustafsson
> On 26 Sep 2021, at 10:57, wangsh.f...@fujitsu.com wrote: > Can we add these file to gitignore? As postgres isn't mandating a specific IDE or dev environment, we typically don't add these files to the .gitignore we ship. If we did it would be an enormous list we'd have to curate and maintain.

can we add some file(msvc) to gitignore

2021-09-26 Thread wangsh.f...@fujitsu.com
Hi, When I work on patch[1] on windows(vs2019), I found there are some file(generated by vs2019) are not listed in gitignore. > $ git status > > Untracked files: > (use "git add ..." to include in what will be committed) >.vs/ >postgres.vcxproj.user >src/tools/msvc/bu

Re: BUG #16583: merge join on tables with different DB collation behind postgres_fdw fails

2021-09-26 Thread Etsuro Fujita
On Sat, Sep 25, 2021 at 10:59 PM Tom Lane wrote: > Etsuro Fujita writes: > > On Sat, Sep 25, 2021 at 4:11 AM Tom Lane wrote: > >> Longer-term, it seems like we really have to be able to represent > >> the notion of a remote column that has an "unknown" collation (that > >> is, one that doesn't m

RE: drop tablespace failed when location contains .. on win32

2021-09-26 Thread wangsh.f...@fujitsu.com
Hi, > -Original Message- > From: Kyotaro Horiguchi > Sent: Monday, September 13, 2021 4:36 PM > To: mich...@paquier.xyz > Mmm. I haven't thought that so seriously, but '/hoge/foo/bar' doesn't > seem to be an absolute path on Windows since it lacks > ":" or "//hostname" part. If we're o