Re: Next Steps with Hash Indexes

On Thu, 14 Oct 2021 at 16:09, Peter Geoghegan wrote: > > On Thu, Oct 14, 2021 at 12:48 AM Simon Riggs > wrote: > > The hash index tuples are 20-bytes each. If that were rounded up to > > 8-byte alignment, then that would be 24 bytes. > > > > Using pageinspect, the max(live_items) on any data page

[PATCH] Make ENOSPC not fatal in semaphore creation

From: Mikhail We might be in situation when we have "just enough" semaphores in the system limit to start but previously crashed unexpectedly, in that case we won't be able to start again - semget() will return ENOSPC, despite the semaphores are ours, and we can recycle them, so check this situat

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

mp39...@gmail.com writes: > We might be in situation when we have "just enough" semaphores in the > system limit to start but previously crashed unexpectedly, in that case > we won't be able to start again - semget() will return ENOSPC, despite > the semaphores are ours, and we can recycle them, so

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

On Sun, Oct 17, 2021 at 10:29:24AM -0400, Tom Lane wrote: > mp39...@gmail.com writes: > > We might be in situation when we have "just enough" semaphores in the > > system limit to start but previously crashed unexpectedly, in that case > > we won't be able to start again - semget() will return ENOS

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

Mikhail writes: > On Sun, Oct 17, 2021 at 10:29:24AM -0400, Tom Lane wrote: >> AFAICS, this patch could be disastrous. What if the semaphore in >> question belongs to some other postmaster? > Does running more than one postmaster on the same PGDATA is supported at > all? Currently seed for the s

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

On Sun, Oct 17, 2021 at 10:52:38AM -0400, Tom Lane wrote: > Mikhail writes: > > On Sun, Oct 17, 2021 at 10:29:24AM -0400, Tom Lane wrote: > >> AFAICS, this patch could be disastrous. What if the semaphore in > >> question belongs to some other postmaster? > > > Does running more than one postmas

Re: lastOverflowedXid does not handle transaction ID wraparound

> On Tue, Oct 12, 2021 at 09:53:22PM -0700, Stan Hu wrote: > > I described how PostgreSQL can enter into a suboverflow condition on > the replica under a number of conditions: > > 1. A long transaction starts. > 2. A single SAVEPOINT is issued. > 3. Many rows are updated on the primary, and the sam

pg_dump does way too much before acquiring table locks

While poking at pg_dump for some work I'll show later, I grew quite unhappy with the extent to which people have ignored this advice given near the head of getTables(): * Note: in this phase we should collect only a minimal amount of * information about each table, basically just enough

Re: Refactoring pg_dump's getTables()

On 2021-Oct-16, Tom Lane wrote: > Attached is a proposed patch that refactors getTables() along the > same lines as some previous work (eg 047329624, ed2c7f65b, daa9fe8a5) > to avoid having multiple partially-redundant copies of the SQL query. > This gets rid of nearly 300 lines of duplicative spa

Re: [PATCH] Proposal for HIDDEN/INVISIBLE column

On 10/17/21 11:01 PM, Gilles Darold wrote: > >   - Add a check into SET UNEXPANDED code to verify that there is at > least one column expanded. What is the point of this? Postgres allows column-less tables. Both of these statements are valid: - CREATE TABLE nada (); - SELECT; -- Vik Fearing

Re: [PATCH] Make ENOSPC not fatal in semaphore creation

On Mon, Oct 18, 2021 at 4:49 AM Mikhail wrote: > The logic works - the initial call to semget() in > InternalIpcSemaphoreCreate returns -1 and errno is set to ENOSPC - I > tested the patch on OpenBSD 7.0, it successfully recycles sem's after > previous "pkill -6 postgres". Verified it with 'ipcs -

Re: XTS cipher mode for cluster file encryption

On 10/16/21 16:16, Bruce Momjian wrote: On Fri, Oct 15, 2021 at 10:57:03PM +0200, Tomas Vondra wrote: That said, I don't think that's really a huge issue or something that's a show stopper or a reason to hold off on using XTS. Note that what those bits actually *are* isn't leaked, just that

Re: XTS cipher mode for cluster file encryption

On 10/16/21 18:28, Bruce Momjian wrote: On Sat, Oct 16, 2021 at 09:15:05AM -0700, Andres Freund wrote: Hi, On 2021-10-16 10:16:25 -0400, Bruce Momjian wrote: As a final comment to Andres's email, adding a GCM has the problems above, plus it wouldn't detect changes to pg_xact, fsm, vm, etc, whi

Re: [PATCH] Proposal for HIDDEN/INVISIBLE column

Le 17/10/2021 à 23:04, Vik Fearing a écrit : > On 10/17/21 11:01 PM, Gilles Darold wrote: >>   - Add a check into SET UNEXPANDED code to verify that there is at >> least one column expanded. > What is the point of this? Postgres allows column-less tables. > > Both of these statements are valid: >

Re: prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619

On Sun, Aug 15, 2021 at 09:44:55AM -0500, Justin Pryzby wrote: > On Sun, May 16, 2021 at 04:23:02PM -0400, Tom Lane wrote: > > 1. Fix FullXidRelativeTo to be a little less trusting. It'd > > probably be sane to make it return FirstNormalTransactionId > > when it'd otherwise produce a wrapped-aroun

Re: [PATCH] Proposal for HIDDEN/INVISIBLE column

On Sun, 17 Oct 2021 at 17:42, Gilles Darold wrote: > Yes, my first though was to allow all columns to be unexpandable like a > table without column, but the the problem is that when you execute > "SELECT * FROM nada" it returns no rows which is not the case of a table > with hidden column. I cou

Re: Refactoring pg_dump's getTables()

Alvaro Herrera writes: > Yeah, this seems a lot better than the original coding. Maybe I would > group together the changes that all require the same version test, > rather than keeping the output columns in the same order. This reduces > the number of branches. Because the follow-on code uses

Re: Skipping logical replication transactions on subscriber side

On Tue, Oct 12, 2021 at 7:58 PM Greg Nancarrow wrote: > > On Tue, Oct 12, 2021 at 4:00 PM Masahiko Sawada wrote: > > > > I've attached updated patches. > > > > Some comments for the v16-0003 patch: Thank you for the comments! > > (1) doc/src/sgml/logical-replication.sgml > > The output from "SE

Re: Skipping logical replication transactions on subscriber side

On Wed, Oct 13, 2021 at 10:59 AM Greg Nancarrow wrote: > > On Tue, Oct 12, 2021 at 4:00 PM Masahiko Sawada wrote: > > > > I've attached updated patches. > > > > Some comments for the v16-0001 patch: > Thank you for the comments! > > src/backend/postmaster/pgstat.c > > (1) pgstat_vacuum_subworke

Re: Skipping logical replication transactions on subscriber side

On Thu, Oct 14, 2021 at 5:45 PM Greg Nancarrow wrote: > > On Tue, Oct 12, 2021 at 4:00 PM Masahiko Sawada wrote: > > > > I've attached updated patches. > > > > A couple more comments for some issues that I noticed in the v16 patches: > > v16-0002 > > doc/src/sgml/ref/alter_subscription.sgml > > (

Re: XTS cipher mode for cluster file encryption

Just a mention. the HMAC (or AE/AD) can be disabled in AES-GCM. HMAC in AES-GCM is an encrypt-then-hash MAC. CRC-32 is not a crypto-safe hash (technically CRC-32 is not a hash function). Cryptographers may unhappy with CRC-32. I think CRC or SHA is not such important. If IV can be stored, I b

Re: XTS cipher mode for cluster file encryption

On 2021/10/16 04:57, Tomas Vondra wrote: > > Seems reasonable, on the assumption the threat models are the same. On 2021/10/16 03:22, Stephen Frost wrote: plain64: the initial vector is the 64-bit little-endian version of the sector number, padded with zeros if necessary That is, the default fo

RE: Failed transaction statistics to measure the logical replication progress

On Thursday, October 14, 2021 2:13 PM Osumi, Takamichi wrote: > On Thursday, October 14, 2021 12:54 PM Hou, Zhijie > wrote: > > On Thursday, September 30, 2021 12:15 PM Amit Kapila > > > On Thu, Sep 30, 2021 at 8:22 AM Hou, Zhijie/侯 志杰wrote: > > > > > > > > On Tues, Sep 28, 2021 6:05 PM Amit Kapil

Re: Data is copied twice when specifying both child and parent table in publication

On Sat, Oct 16, 2021 at 5:30 PM houzj.f...@fujitsu.com wrote: > > On Friday, October 15, 2021 7:23 PM houzj.f...@fujitsu.com wrote: > > Attach a patch to fix it. > Attach a new version patch which refactor the fix code in a cleaner way. > I have not debugged it yet to find out why, but with the p

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

> Add a test case in subscription/t/100_bugs.pl. Please consider the v2 patch > for review. > Reviewed and tested the patch, it works fine... There are some trailing spaces present in the newly added code lines, which needs to be corrected... Doing some further testing with different datestyles,

Re: Reset snapshot export state on the transaction abort

On Sun, Oct 17, 2021 at 09:33:48AM +0900, Michael Paquier wrote: > That seems logically fine. I'll check that tomorrow. And that looks indeed fine. I have adjusted a couple of things, and backpatched the fix. -- Michael signature.asc Description: PGP signature

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

On Thu, Oct 14, 2021 at 8:50 PM Dilip Kumar wrote: > > On Thu, Oct 14, 2021 at 3:48 PM Sadhuprasad Patro wrote: > > > > Hi All, > > > > Publisher 'DateStyle' is set as "SQL, MDY", whereas in Subscriber as > > "SQL, DMY", the logical replication is not working... > > > > From Publisher: > > postgr

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

Masahiko Sawada writes: > On Thu, Oct 14, 2021 at 8:50 PM Dilip Kumar wrote: >> Looks like a problem to me, I think for fixing this, on logical >> replication connection always set subscriber's DateStlyle, with that >> the walsender will always send the data in the same DateStyle that >> worker u

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

I wrote: > An alternative that wouldn't require a network round trip is for the > publisher to set its own datestyle to ISO/YMD. I'm pretty sure that > will be interpreted correctly regardless of the receiver's datestyle. Ah ... see postgres_fdw's set_transmission_modes(). I think we want to cop

Re: Added schema level support for publication.

On Fri, Oct 15, 2021 at 6:45 AM Greg Nancarrow wrote: > > On Thu, Oct 14, 2021 at 9:59 PM Amit Kapila wrote: > > > > > If partitions belong to a different schema than the parent partitioned > > > table, then the current patch implementation allows the partitions to > > > (optionally) be explicitl

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

On Sun, Oct 17, 2021 at 11:41:35PM -0400, Tom Lane wrote: > Ah ... see postgres_fdw's set_transmission_modes(). I think we want > to copy that logic not invent some other way to do it. dblink.c has something similar as of applyRemoteGucs(), except that it does not do extra_float_digits. It would

Re: Added schema level support for publication.

On Sat, Oct 16, 2021 at 4:57 PM houzj.f...@fujitsu.com wrote: > > Besides, I found we misunderstood the flag PUBLICATION_PART_ROOT it means: > "ROOT: only the table explicitly mentioned in the publication" We cannot use > it > as a flag to judge whether do the partition filtering, I think we need

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

On Mon, 18 Oct 2021 at 11:14, Sadhuprasad Patro wrote: >> Add a test case in subscription/t/100_bugs.pl. Please consider the v2 patch >> for review. >> > > Reviewed and tested the patch, it works fine... There are some > trailing spaces present in the newly added code lines, which needs to > be

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

On Mon, 18 Oct 2021 at 11:59, Michael Paquier wrote: > On Sun, Oct 17, 2021 at 11:41:35PM -0400, Tom Lane wrote: >> Ah ... see postgres_fdw's set_transmission_modes(). I think we want >> to copy that logic not invent some other way to do it. > > dblink.c has something similar as of applyRemoteG

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

On Mon, 18 Oct 2021 at 11:41, Tom Lane wrote: > I wrote: >> An alternative that wouldn't require a network round trip is for the >> publisher to set its own datestyle to ISO/YMD. I'm pretty sure that >> will be interpreted correctly regardless of the receiver's datestyle. > > Ah ... see postgre

Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber

Japin Li writes: > On Mon, 18 Oct 2021 at 11:59, Michael Paquier wrote: >> dblink.c has something similar as of applyRemoteGucs(), except that it >> does not do extra_float_digits. It would be nice to avoid more >> duplication for those things, at least on HEAD. On the top of my >> head, don't

Re: prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619

On Sun, Oct 17, 2021 at 04:43:15PM -0500, Justin Pryzby wrote: > On Sun, Aug 15, 2021 at 09:44:55AM -0500, Justin Pryzby wrote: > > On Sun, May 16, 2021 at 04:23:02PM -0400, Tom Lane wrote: > > > 1. Fix FullXidRelativeTo to be a little less trusting. It'd > > > probably be sane to make it return F

Re: pgsql: Document XLOG_INCLUDE_XID a little better

On Thu, Oct 7, 2021 at 10:46 AM Amit Kapila wrote: > > On Sat, Oct 2, 2021 at 4:16 PM Dilip Kumar wrote: > > > > On Fri, Oct 1, 2021 at 6:24 PM Alvaro Herrera > > wrote: > > > > > > On 2021-Oct-01, Amit Kapila wrote: > > > > > I think a straight standalone variable (probably a static boolean in

Re: using an end-of-recovery record in all cases

On Wed, Oct 6, 2021 at 7:24 PM Amul Sul wrote: > > On Tue, Oct 5, 2021 at 10:42 PM Robert Haas wrote: > > > > On Tue, Oct 5, 2021 at 12:41 PM Amul Sul wrote: > > > No, InRecovery flag get cleared before this point. I think, we can use > > > lastReplayedEndRecPtr what you have suggested in other

Re: can we add subscription TAP test option "vcregress subscriptioncheck" for MSVC builds?

On Sat, Oct 16, 2021 at 6:35 PM Andrew Dunstan wrote: > > > The problems with having "vcregress checkworld" are: 1) required code > > modifications are more as the available "vcregress" test functions, > > which required pre-started pg instance, can't be used directly. 2) it > > looks like spinnin

try_relation_open and relation_open behave different.

Hi hackers, I'm writing an extension that employs `object_access_hook`. I want to monitor the table creation event and record the mapping between `reloid` and `relfilenode` during a transaction. Here's my code snippet, ``` static void my_object_access_hook(ObjectAccessType access,

Re: Data is copied twice when specifying both child and parent table in publication

On Mon, Oct 18, 2021 at 8:27 AM Greg Nancarrow wrote: > > On Sat, Oct 16, 2021 at 5:30 PM houzj.f...@fujitsu.com > wrote: > > > > On Friday, October 15, 2021 7:23 PM houzj.f...@fujitsu.com wrote: > > > Attach a patch to fix it. > > Attach a new version patch which refactor the fix code in a clean

RE: Added schema level support for publication.

On Monday, October 18, 2021 12:04 PM Greg Nancarrow wrote: > On Sat, Oct 16, 2021 at 4:57 PM houzj.f...@fujitsu.com wrote: > > Based on the V40 patchset, attaching the Top-up patch which try to fix > > the partition issue in a cleaner way. > > > > A minor thing, in your "top-up patch", the test co

Re: [PATCH] Proposal for HIDDEN/INVISIBLE column

Le 17/10/2021 à 23:48, Isaac Morland a écrit : > On Sun, 17 Oct 2021 at 17:42, Gilles Darold > wrote: > > Perhaps I misunderstand what you are saying, but a no-columns table > definitely can return rows: > > psql (12.2) > Type "help" for help. > > postgres=# create table

Re: try_relation_open and relation_open behave different.

On Mon, Oct 18, 2021 at 01:56:07PM +0800, Xing GUO wrote: > My question is, is it a deliberate design that makes try_relation_open and > relation_open different? Shall we mention it in the comment of > try_relation_open OR adding the checker to relation_open? I am not sure what you mean here, both

Re: [BUG] Unexpected action when publishing partition tables

On Wed, Oct 13, 2021 at 9:11 AM Amit Langote wrote: > > Hi Amit, > > On Fri, Oct 8, 2021 at 12:47 PM Amit Kapila wrote: > > On Thu, Oct 7, 2021 at 12:39 PM Amit Langote > > wrote: > > > Sorry that I didn't comment on this earlier, but I think either > > > GetPubPartitionOptionRelations() or Inv

Re: [BUG] Unexpected action when publishing partition tables

On Mon, Oct 18, 2021 at 12:24 PM Amit Kapila wrote: > > On Wed, Oct 13, 2021 at 9:11 AM Amit Langote wrote: > > Anyway, I'll think a bit more about the possible hazards of not doing > > the locking and will reply again if there's indeed a problem(s) that > > needs to be fixed. > > > > I think you