Re: Document atthasmissing default optimization avoids verification table scan

2022-03-26 Thread David G. Johnston
On Sat, Mar 26, 2022 at 3:25 PM James Coleman wrote: > On Fri, Mar 25, 2022 at 4:40 PM Robert Haas wrote: > > > > On Tue, Jan 25, 2022 at 8:49 AM James Coleman wrote: > > > Here's a version that looks like that. I'm not convinced it's an > > > improvement over the previous version: again, I

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 9:00 PM Mark Dilger wrote: > A grant or revoke on an unrecognized custom parameter will create a SUSET > placeholder, [...] > which cleans up the problem, with one exception: if the user executes a > "revoke set on parameter some.such from public" prior to loading the

Re: Document atthasmissing default optimization avoids verification table scan

2022-03-27 Thread David G. Johnston
On Sun, Mar 27, 2022 at 10:00 AM James Coleman wrote: > As shown above, table scans (and specifically table scans used to > validate constraints, which is what this patch is about) are clearly > documented (more than once!) in the ALTER TABLE documentation. In fact > it's documented specifically

Re: Document atthasmissing default optimization avoids verification table scan

2022-03-27 Thread David G. Johnston
On Sun, Mar 27, 2022 at 11:17 AM James Coleman wrote: > Hmm, I didn't realize that was project policy, Guideline/Rule of Thumb is probably a better concept. > but I'm a bit > surprised given that the sentence which 0001 replaces seems like a > direct violation of that also: "In neither case

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-30 Thread David G. Johnston
On Wed, Mar 30, 2022 at 8:12 AM Andrew Dunstan wrote: > > On 3/30/22 09:26, Tom Lane wrote: > > > > > What this loses is the ability to revoke public SET permissions > > on USERSET GUCs. I claim that that is not so valuable as to > > justify all the complication needed to deal with it.

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-30 Thread David G. Johnston
On Wed, Mar 30, 2022 at 8:46 AM Tom Lane wrote: > I don't want to do that with > a blunderbuss, but perhaps there's an argument to do it for specific > cases (search_path comes to mind, though the performance cost could be > significant, since I think setting that in function SET clauses is >

Re: Returning multiple rows in materialized mode inside the extension

2022-03-30 Thread David G. Johnston
On Wed, Mar 30, 2022 at 9:01 AM Piotr Styczyński wrote: > I don’t know if this mailing list is a good place to ask this question, > but if it’s not, just correct me. > pgsql-general is probably better > *The problem:* > > We currently have a one-to-many function (an operation that produces >

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-30 Thread David G. Johnston
On Wednesday, March 30, 2022, Andres Freund wrote: > > My current proposal is to just have two reset times. One for the contents > of > pg_stat_database (i.e. not affected by pg_stat_reset_single_*_counters()), > and > one for stats within the entire database. > > What IS it affected by? And

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-30 Thread David G. Johnston
On Wed, Mar 30, 2022 at 1:39 PM Andres Freund wrote: > Hi, > > On 2022-03-30 12:29:51 -0700, David G. Johnston wrote: > > On Wednesday, March 30, 2022, Andres Freund wrote: > > > My current proposal is to just have two reset times. One for the > contents > >

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 4:43 PM Andres Freund wrote: > But more importantly, a > per-relation/function reset field wouldn't address Tomas's concern: He > wants a > single thing to check to see if any stats have been reset - and that's imo > a > quite reasonable desire. > Per the original

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 5:20 PM Peter Geoghegan wrote: > On Tue, Mar 29, 2022 at 4:55 AM Michail Nikolaev > wrote: > > I think that you could do a better job of explaining and promoting the > problem that you're trying to solve here. Emphasis on the problem, not > so much the solution. As a

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 5:56 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Tue, Mar 29, 2022 at 5:50 PM Andres Freund wrote: > >> Hi, >> >> On 2022-03-29 17:06:24 -0700, David G. Johnston wrote: >> > On Tue, Mar 29, 2022 at 4:43 PM A

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 1:37 PM Andres Freund wrote: > > Secondly, to do anything really meaningful you need to calculate deltas, > > and be able to detect if some of the stats were reset for the particular > > interval. And the stat_reset timestamp was designed to be a simple way > > to detect

Re: Temporary tables versus wraparound... again

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 4:52 PM Greg Stark wrote: > On Mon, 28 Mar 2022 at 16:30, Andres Freund wrote: > > > > > Make ON COMMIT DELETE ROWS reset relfrozenxmin and other table > stats > > > like normal truncate. Otherwise even typical short-lived > transactions > > > using temporary

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 5:50 PM Andres Freund wrote: > Hi, > > On 2022-03-29 17:06:24 -0700, David G. Johnston wrote: > > On Tue, Mar 29, 2022 at 4:43 PM Andres Freund > wrote: > > > But more importantly, a > > > per-relation/function reset field wouldn't add

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2022-03-29 Thread David G. Johnston
On Tue, Mar 22, 2022 at 6:52 AM Michail Nikolaev wrote: > Hello, Andres. > > > Fails to apply at the moment: http://cfbot.cputube.org/patch_37_2947.log > > Thanks for notifying me. BTW, some kind of automatic email in case of > status change could be very helpful. > > > Marked as waiting for

Re: unlogged sequences

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 9:28 AM Andres Freund wrote: > I agree it makes sense to have logged sequences with unlogged tables. We > should call out the behavioural change somewhere prominent in the release > notes. > > We can/do already support that unlikely use case by allowing one to remove the

Re: unlogged sequences

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 12:36 PM Tomas Vondra wrote: > On 3/31/22 19:35, David G. Johnston wrote: > > On Thu, Mar 31, 2022 at 9:28 AM Andres Freund > <mailto:and...@anarazel.de>> wrote: > > > > I agree it makes sense to have logged sequences with unlogged

Re: [WIP] ALTER COLUMN IF EXISTS

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 4:39 PM Bradley Ayers wrote: > > I'm interested in adding more ergonomics to DDL commands, in > particular supporting IF EXISTS for ALTER TABLE … ALTER COLUMN, so > that if a column doesn't exist the command is skipped. > > IF EXISTS is already supported in various places

Re: unlogged sequences

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 1:05 PM Tomas Vondra wrote: > > I agree the first part is not contentious, so shall we extract this part > of the patch and get that committed for PG15? Or is that too late to > make such changes to the patch? > > The minimum viable feature for me, given the written goal

Re: unlogged sequences

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 3:43 PM Tomas Vondra wrote: > On 3/31/22 22:40, David G. Johnston wrote: > > On Thu, Mar 31, 2022 at 1:05 PM Tomas Vondra > > mailto:tomas.von...@enterprisedb.com>> > > wrote: > > > > > > I agree the first part is not

Re: unlogged sequences

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 5:25 PM Robert Haas wrote: > On Thu, Mar 31, 2022 at 10:14 AM Tomas Vondra > wrote: > > * When linking a sequence to a table (ALTER SEQUENCE ... OWNED BY), > > there's an ereport(ERROR) if the relpersistence values do not match. > > > > * Disallow changing persistence

Re: unlogged sequences

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 6:03 PM Robert Haas wrote: > > In this new system, does the user still get a logged sequence? If they > get an unlogged sequence, how does dump-and-restore work? What if they > want to still have a logged sequence? But for sequences that are > simply owned, there is no

Re: unlogged sequences

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 1:40 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > The DBA just has to execute the ALTER SEQUENCE command on all relevant > sequences. > Additional, if we do not implement the forced matching of persistence mode, we should consider adding an &

Re: Higher level questions around shared memory stats

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 7:33 PM Kyotaro Horiguchi wrote: > At Thu, 31 Mar 2022 14:04:16 -0700, Andres Freund > wrote in > > Hi, > > > > On 2022-03-31 16:16:31 +0900, Kyotaro Horiguchi wrote: > > > After moving to shared stats, we might want to expose the GUC variable > > > itself. Then

Re: unlogged sequences

2022-03-31 Thread David G. Johnston
On Thu, Mar 31, 2022 at 6:03 PM Robert Haas wrote: > On Thu, Mar 31, 2022 at 8:44 PM David G. Johnston > wrote: > > > The "give the user power" argument is also valid. But since they > already have power through unowned sequences, having the owned sequences >

Re: unlogged sequences

2022-04-01 Thread David G. Johnston
On Fri, Apr 1, 2022 at 9:22 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > > On 01.04.22 00:43, Tomas Vondra wrote: > > Hmm, so what about doing a little bit different thing: > > > > 1) owned sequences inherit persistence of the table by default > > > > 2) allow ALTER SEQUENCE

Re: unlogged sequences

2022-04-01 Thread David G. Johnston
On Fri, Apr 1, 2022 at 9:31 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 01.04.22 18:22, Peter Eisentraut wrote: > > > > On 01.04.22 00:43, Tomas Vondra wrote: > >> Hmm, so what about doing a little bit different thing: > >> > >> 1) owned sequences inherit persistence of

Re: unlogged sequences

2022-04-03 Thread David G. Johnston
On Sun, Apr 3, 2022 at 12:36 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 03.04.22 20:50, David G. Johnston wrote: > > However, tables having an identity sequence seem to be unaddressed in > > this patch. The existing (and unchanged) p

Re: unlogged sequences

2022-04-03 Thread David G. Johnston
On Sun, Apr 3, 2022 at 12:36 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 03.04.22 20:50, David G. Johnston wrote: > > However, tables having an identity sequence seem to be unaddressed in > > this patch. The existing (and unchanged) p

Re: unlogged sequences

2022-04-03 Thread David G. Johnston
On Sun, Apr 3, 2022 at 10:19 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > Here is an updated patch that fixes this pg_dump/pg_upgrade issue and > also adds a few more comments and documentation sentences about what > happens and what is allowed. I didn't change any

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Sun, Apr 3, 2022 at 9:16 PM Andres Freund wrote: > > Please take a look! > > I didn't take the time to fixup all the various odd typos in the general code comments; none of them reduced comprehension appreciably. I may do so when/if I do another pass. I did skim over the entire patch set

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Mon, Apr 4, 2022 at 2:54 PM Andres Freund wrote: > > > As the existing function only handles functions and relations why not > just > > perform a specific Kind check for them? Generalizing to assert on > whether > > or not the function works on fixed or variable Kinds seems beyond its > >

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Mon, Apr 4, 2022 at 2:06 PM Andres Freund wrote: > > > My first encounter with pg_stat_exists_stat() didn't draw my attention as > > being problematic so I'd say we just stick with it. As a SQL user > reading: > > WHERE exists (...) is somewhat natural; using "have" or back-to-back > >

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Mon, Apr 4, 2022 at 3:44 PM Andres Freund wrote: > Hi, > > On 2022-04-04 15:24:24 -0700, David G. Johnston wrote: > > Replacing the existing assert(!kind->fixed_amount) with > > assert(!kind->accessed_across_databases) produces the same result as the > > l

Re: shared-memory based stats collector - v68

2022-04-05 Thread David G. Johnston
On Mon, Apr 4, 2022 at 7:36 PM Andres Freund wrote: > > I think all this is going to achieve is to making code more complicated. > There > is a *single* non-assert use of accessed_across_databases and now a single > assertion involving it. > > What would having PGSTAT_KIND_CLUSTER and

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread David G. Johnston
On Mon, Jan 24, 2022 at 10:49 PM Michael Paquier wrote: > What about patterns? Switches like --table or > --extension are able to digest a psql-like pattern to decide which > objects to dump. > The extension object type does not seem to have gotten the --exclude-extension capability that it

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread David G. Johnston
On Thu, Mar 24, 2022 at 5:40 PM Tom Lane wrote: > "David G. Johnston" writes: > > The extension object type does not seem to have gotten the > > --exclude-extension capability that it would need to conform to the > general > > design exemplified by

Re: Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-03-24 Thread David G. Johnston
On Thu, Mar 24, 2022 at 4:42 PM Chapman Flack wrote: > On 03/27/21 08:57, Andrew Dunstan wrote: > > We can bikeshed the name of the flag at some stage. --procedures-only > > might also make sense > > Any takers for --routines-only ? > > "Routine" is the genuine, ISO SQL umbrella term for a

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-23 Thread David G. Johnston
On Wed, Mar 23, 2022 at 5:55 PM Andres Freund wrote: > > Starting with the below commit, pg_stat_reset_single_function_counters, > pg_stat_reset_single_table_counters don't just reset the stats for the > individual function, but also set pg_stat_database.stats_reset. > > commit

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-01 Thread David G. Johnston
On Tue, Feb 1, 2022 at 11:55 PM Amit Kapila wrote: > On Wed, Feb 2, 2022 at 9:41 AM David G. Johnston > wrote: > > > > On Tue, Feb 1, 2022 at 8:07 PM Amit Kapila > wrote: > >> > >> On Tue, Feb 1, 2022 at 11:47 AM Masahiko Sawada > wrote: > &

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-01 Thread David G. Johnston
On Tue, Feb 1, 2022 at 8:07 PM Amit Kapila wrote: > On Tue, Feb 1, 2022 at 11:47 AM Masahiko Sawada > wrote: > > > > > I see that it's better to use a better IPC for ALTER SUBSCRIPTION SKIP > > feature to pass error-XID or error-LSN information to the worker > > whereas I'm also not sure of the

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-02 Thread David G. Johnston
On Wed, Feb 2, 2022 at 5:08 AM Amit Kapila wrote: > On Wed, Feb 2, 2022 at 1:06 PM David G. Johnston > wrote: > > ... > > > > I already explained that the concept of err_cnt is not useful. The fact > that you include it here makes me think you are still think

Re: warn if GUC set to an invalid shared library

2022-02-02 Thread David G. Johnston
On Tue, Feb 1, 2022 at 11:06 PM Maciek Sakrejda wrote: > I tried running ALTER SYSTEM and got the warnings as expected: > > postgres=# alter system set shared_preload_libraries = > no_such_library,not_this_one_either; > WARNING: could not access file "$libdir/plugins/no_such_library" > WARNING:

Re: Unclear problem reports

2022-02-02 Thread David G. Johnston
On Wed, Feb 2, 2022 at 5:35 PM Bruce Momjian wrote: > I consider these as problems that need digging to find the cause, and > users are usually unable to do sufficient digging, and we don't have > time to give them instructions, so they never get a reply. > > Is there something we can do to

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-02 Thread David G. Johnston
On Wednesday, February 2, 2022, Masahiko Sawada wrote: > and have other error > information in pg_stat_subscription_workers view. > What benefit is there to keeping the existing collector-based pg_stat_subscripiton_workers view? If we re-write it using shmem IPC then we might as well put

Re: Design of pg_stat_subscription_workers vs pgstats

2022-01-27 Thread David G. Johnston
On Thu, Jan 27, 2022 at 5:08 AM Amit Kapila wrote: > On Thu, Jan 27, 2022 at 11:16 AM Andres Freund wrote: > > > > On 2022-01-25 20:27:07 +0900, Masahiko Sawada wrote: > > > > > There will be some challenges in a case where updating > pg_subscription_rel > > > also failed too (what to report to

Re: substring odd behavior

2022-01-27 Thread David G. Johnston
On Thu, Jan 27, 2022 at 7:22 PM Regina Obe wrote: > Is this intentional behavior? > > -- I can do this > SELECT substring('3.2.0' from '[0-9]*\.([0-9]*)\.'); > > -- But can't do this gives error syntax error at or near "from" > SELECT pg_catalog.substring('3.2.0' from '[0-9]*\.([0-9]*)\.'); > >

Re: Design of pg_stat_subscription_workers vs pgstats

2022-01-27 Thread David G. Johnston
On Thu, Jan 27, 2022 at 2:15 PM Andres Freund wrote: > Another related thing is that using a 32bit xid for allowing skipping is a > bad > idea anyway - we shouldn't adding new interfaces with xid wraparound > dangers - > it's getting more and more common to have multiple wraparounds a day. An >

Re: Output clause for Upsert aka INSERT...ON CONFLICT

2022-01-26 Thread David G. Johnston
On Wednesday, January 26, 2022, Anand Sowmithiran wrote: > The INSERT...ON CONFLICT is used for doing upserts in one of our app. > Our app works with both MS SQL and Postgresql, based on customer needs. > > Unlike the MS SQL MERGE command's OUTPUT clause that gives the $action >

Re: Storage for multiple variable-length attributes in a single row

2022-02-07 Thread David G. Johnston
On Mon, Feb 7, 2022 at 9:58 AM Esteban Zimanyi wrote: > > As suggested by David, this goes beyond the "traditional" usage of > PostgreSQL. Therefore my questions are > * What is the suggested strategy to splitting these 2K attributes into > vertically partitioned tables where the tables are

Re: Storage for multiple variable-length attributes in a single row

2022-02-07 Thread David G. Johnston
On Mon, Feb 7, 2022 at 8:44 AM Esteban Zimanyi wrote: > May I kindly ask your insight about a question I posted 1 month ago and > for which I never received any answer ? > -hackers really isn't the correct place for usage questions like this - even if you are creating a custom type (why you are

Re: Output clause for Upsert aka INSERT...ON CONFLICT

2022-01-27 Thread David G. Johnston
On Thursday, January 27, 2022, Anand Sowmithiran wrote: > > However, the MS SQL server MERGE command also does 'delete' using the > 'when not matched' clause, is there an equivalent ? > PostgreSQL does not have a merge command feature. Just the subset of behavior that is INSERT…on conflict

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 11:55 AM James Coleman wrote: > On Thu, Jan 20, 2022 at 3:43 PM James Coleman wrote: > > > > As noted earlier I expect to be posting an updated patch soon. > > Here's the updated series. In 0001 I've moved the documentation tweak > into the ALTER TABLE notes section. In

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 2:08 PM Andrew Dunstan wrote: > On 1/21/22 13:55, James Coleman wrote: > > + Before PostgreSQL 11, adding a new > column to a > + table required rewriting that table, making it a very slow operation. > + More recent versions can sometimes optimize away this

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 4:55 AM Amit Kapila wrote: > Apart from this, I have changed a few comments and ran pgindent. Do > let me know what you think of the changes? > The paragraph describing ALTER SUBSCRIPTION SKIP seems unnecessarily repetitive. Consider: """ Skips applying all changes of

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 2:50 PM Tom Lane wrote: > "David G. Johnston" writes: > > On Fri, Jan 21, 2022 at 2:08 PM Andrew Dunstan > wrote: > >> I know what it's replacing refers to release 11, but let's stop doing > >> that. How about something like

Re: Proposal: allow database-specific role memberships

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 3:12 PM Kenaniah Cerny wrote: > The latest rebased version of the patch is attached. > As I was just reminded, we tend to avoid specifying specific PostgreSQL versions in our documentation. We just say what the current version does. Here, the note sentences at lines 62

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 5:14 PM James Coleman wrote: > > > Really? That's horrid, because that's directly useful advice. > > Remedied, but rewritten a bit to better fit with the new style/goal of > that tip). > > Version 3 is attached. > > Coming back to this after a respite I think the tip

Re: Skipping logical replication transactions on subscriber side

2022-01-21 Thread David G. Johnston
On Fri, Jan 21, 2022 at 10:30 PM Amit Kapila wrote: > On Fri, Jan 21, 2022 at 10:00 PM David G. Johnston > wrote: > > > > On Fri, Jan 21, 2022 at 4:55 AM Amit Kapila > wrote: > >> > >> Apart from this, I have changed a few comments and ran pgind

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-19 Thread David G. Johnston
On Wed, Jan 19, 2022 at 5:08 PM Bossart, Nathan wrote: > On 9/24/21, 7:30 AM, "James Coleman" wrote: > > When PG11 added the ability for ALTER TABLE ADD COLUMN to set a constant > > default value without rewriting the table the doc changes did not note > > how the new feature interplayed with

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-19 Thread David G. Johnston
On Wed, Jan 19, 2022 at 6:14 PM James Coleman wrote: > I'm open to the idea of wordsmithing here, of course, but I strongly > disagree that this is irrelevant data. Ok, but wording aside, only changing a tip in the DDL - Add Table section doesn't seem like a complete fix. The notes in alter

Re: Document atthasmissing default optimization avoids verification table scan

2022-01-22 Thread David G. Johnston
On Saturday, January 22, 2022, James Coleman wrote: > On Sat, Jan 22, 2022 at 12:35 AM David G. Johnston > wrote: > > > > On Fri, Jan 21, 2022 at 5:14 PM James Coleman wrote: > >> > >> > >> > Really? That's horrid, because that's directly useful

Re: Skipping logical replication transactions on subscriber side

2022-01-22 Thread David G. Johnston
On Sat, Jan 22, 2022 at 2:41 AM Amit Kapila wrote: > On Sat, Jan 22, 2022 at 12:41 PM David G. Johnston > wrote: > > > > On Fri, Jan 21, 2022 at 10:30 PM Amit Kapila > wrote: > >> > >> On Fri, Jan 21, 2022 at 10:00 PM David G. Johnston > >> wrot

Re: Skipping logical replication transactions on subscriber side

2022-01-22 Thread David G. Johnston
On Sat, Jan 22, 2022 at 9:21 AM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Sat, Jan 22, 2022 at 2:41 AM Amit Kapila > wrote: > >> >> > Additionally, the description for pg_stat_subscription_workers should >> describe what happ

Re: Bogus duplicate command issued in pg_dump

2022-01-23 Thread David G. Johnston
On Sun, Jan 23, 2022 at 7:25 PM Michael Paquier wrote: > On Sun, Jan 23, 2022 at 01:31:03PM -0500, Tom Lane wrote: > > We could consider a more global change to get rid of using > > appendPQExpBuffer where it's not absolutely necessary, so that > > there are fewer bad examples to copy. Another

Re: Bogus duplicate command issued in pg_dump

2022-01-23 Thread David G. Johnston
On Sun, Jan 23, 2022 at 11:31 AM Tom Lane wrote: > > res = ExecuteSqlQueryForSingleRow(fout, upgrade_query->data); > ... > appendPQExpBuffer(upgrade_query, > "SELECT t.oid, t.typarray " > ... > res =

Re: Skipping logical replication transactions on subscriber side

2022-01-23 Thread David G. Johnston
On Sun, Jan 23, 2022 at 8:35 PM Amit Kapila wrote: > > I really dislike the user experience this provides, and given it is new > in v15 (and right now this table seems to exist solely to support this > feature) changing this seems within the realm of possibility. I have to > imagine these

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread David G. Johnston
On Sun, Jan 23, 2022 at 11:55 PM Masahiko Sawada wrote: > On Mon, Jan 24, 2022 at 1:49 PM David G. Johnston > wrote: > > > > On Sun, Jan 23, 2022 at 8:35 PM Amit Kapila > wrote: > >> > >> > I really dislike the user experience this provides, a

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread David G. Johnston
On Monday, January 24, 2022, Amit Kapila wrote: > On Mon, Jan 24, 2022 at 1:30 PM David G. Johnston > wrote: > > > > That said, at present my two dislikes: > > > > 1) ALTER SYSTEM SKIP accepts any xid value (I need to consider further > the timing of when thi

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread David G. Johnston
On Tue, Jan 25, 2022 at 7:47 AM Masahiko Sawada wrote: > Yeah, I think it's a good idea to clear the subskipxid after the first > transaction regardless of whether the worker skipped it. > > So basically instead of stopping the worker with an error you suggest having the worker continue applying

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread David G. Johnston
On Tue, Jan 25, 2022 at 8:09 AM Masahiko Sawada wrote: > On Tue, Jan 25, 2022 at 11:58 PM David G. Johnston > wrote: > > > > On Tue, Jan 25, 2022 at 7:47 AM Masahiko Sawada > wrote: > >> > >> Yeah, I think it's a good idea to clear the subskipxid afte

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread David G. Johnston
On Tue, Jan 25, 2022 at 5:52 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 25.01.22 06:18, Amit Kapila wrote: > > I think to avoid this we can send a message to clear this (at least to > > clear XID in the view) after skipping the xact but there is no > > guarantee that it

Re: Undocumented error

2022-01-14 Thread David G. Johnston
On Friday, January 14, 2022, Tomas Vondra wrote: > , > > On 1/14/22 00:02, Petar Dambovaliev wrote: > >> >> the error code is `-1` and the error text is `invalid ordering of >> speculative insertion changes` >> > > Which Postgres version is this, exactly? Was the WAL generated by that > same

Re: JSONB docs patch

2022-01-25 Thread David G. Johnston
On Tue, Jan 25, 2022 at 3:38 PM Mikhail Dobrinin wrote: > Hello, > > I have come across some missing documentation that I think could benefit > the community. > > Several functions like `jsonb_exists`, `jsonb_exists_any`, > `jsonb_exists_all` have existed for many PG versions but were not >

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread David G. Johnston
On Tue, Jan 25, 2022 at 8:33 AM Masahiko Sawada wrote: > Given that we cannot use rely on the pg_stat_subscription_workers view > for this purpose, we would need either a new sub-system that tracks > each logical replication status so the system can set the error XID to > subskipxid, or to wait

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread David G. Johnston
On Mon, Jan 24, 2022 at 12:59 AM David G. Johnston < david.g.johns...@gmail.com> wrote: > > > 5(out). wait for the user to manually restart the replication stream >> >> Do you mean that there always is user intervention after error so the >> replication str

Re: Skipping logical replication transactions on subscriber side

2022-01-25 Thread David G. Johnston
On Tue, Jan 25, 2022 at 9:16 PM Amit Kapila wrote: > On Wed, Jan 26, 2022 at 9:36 AM Masahiko Sawada > wrote: > > On Wed, Jan 26, 2022 at 12:54 PM Amit Kapila > wrote: > > > > > > > > > Probably, we also need to consider the case where the tablesync > worker > > > > entered an error loop and

Re: shared-memory based stats collector - v69

2022-04-05 Thread David G. Johnston
On Tue, Apr 5, 2022 at 2:23 PM Andres Freund wrote: > > On 2022-04-05 13:51:12 -0700, David G. Johnston wrote: > > >, but rather add to the shared queue > > Queue? Maybe you mean the hashtable? > Queue implemented by a list...? Anyway, I think I mean this: /* *

Re: shared-memory based stats collector - v68

2022-04-05 Thread David G. Johnston
On Tuesday, April 5, 2022, Andres Freund wrote: > Hi, > > On 2022-04-05 08:49:36 -0700, David G. Johnston wrote: > > On Mon, Apr 4, 2022 at 7:36 PM Andres Freund wrote: > > > > > > > > I think all this is going to achieve is to making code more > compli

Re: shared-memory based stats collector - v69

2022-04-05 Thread David G. Johnston
On Mon, Apr 4, 2022 at 8:05 PM Andres Freund wrote: > - added an architecture overview comment to the top of pgstat.c - not sure > if > it makes sense to anybody but me (and perhaps Horiguchi-san)? > > I took a look at this, diff attached. Some typos and minor style stuff, plus trying to

Re: shared-memory based stats collector - v69

2022-04-05 Thread David G. Johnston
On Tue, Apr 5, 2022 at 4:16 PM Andres Freund wrote: > On 2022-04-05 14:43:49 -0700, David G. Johnston wrote: > > On Tue, Apr 5, 2022 at 2:23 PM Andres Freund wrote: > > > > > > I guess I should add a paragraph about snapshots / fetch consistency. > > > &g

Re: shared-memory based stats collector - v70

2022-04-05 Thread David G. Johnston
On Tue, Apr 5, 2022 at 8:00 PM Andres Freund wrote: > > Here comes v70: > > I think this is basically ready, minus a a few comment adjustments here and > there. Unless somebody protests I'm planning to start pushing things > tomorrow > morning. > > Nothing I've come across, given my area of

Re: shared-memory based stats collector - v70

2022-04-05 Thread David G. Johnston
On Tue, Apr 5, 2022 at 8:11 PM Greg Stark wrote: > I've never tried to review a 24-patch series before. It's kind of > intimidating Is there a good place to start to get a good idea of > the most important changes? > It isn't as bad as the number makes it sound - I just used "git am" to

Re: shared-memory based stats collector - v69

2022-04-05 Thread David G. Johnston
On Tue, Apr 5, 2022 at 8:14 PM Andres Freund wrote: > > On 2022-04-05 20:00:50 -0700, David G. Johnston wrote: > > * Statistics are loaded from the filesystem during startup (by the startup > * process), unless preceded by a crash, in which case all stats are > * discarded

Re: How about a psql backslash command to show GUCs?

2022-04-07 Thread David G. Johnston
On Thu, Apr 7, 2022 at 9:58 AM Joe Conway wrote: > On 4/7/22 12:37, Tom Lane wrote: > > Mark Dilger writes: > >>> On Apr 7, 2022, at 9:29 AM, Tom Lane wrote: > >>> I wouldn't > >>> fight too hard if people want to lengthen it to \dconfig for > consistency > >>> with set_config(). > > > >> I'd

Re: How about a psql backslash command to show GUCs?

2022-04-07 Thread David G. Johnston
On Thu, Apr 7, 2022 at 7:56 AM Tom Lane wrote: > "Jonathan S. Katz" writes: > > > Maybe to appeal to all crowds, we say "list configuration parameters > > (GUCs)"? > > I'm in the camp that says that GUC is not an acronym we wish to expose > to end users. > > I am too. In any case, either go

Re: shared-memory based stats collector - v70

2022-04-07 Thread David G. Johnston
On Thu, Apr 7, 2022 at 7:10 PM Kyotaro Horiguchi wrote: > At Thu, 7 Apr 2022 16:37:51 -0700, Andres Freund > wrote in > > Hi, > > > > On 2022-04-07 00:28:45 -0700, Andres Freund wrote: > > > I've gotten through the main commits (and then a fix for the apparently > > > inevitable bug that's

Re: shared-memory based stats collector - v70

2022-04-07 Thread David G. Johnston
On Thu, Apr 7, 2022 at 8:59 PM Andres Freund wrote: > > >Cumulative statistics are collected in shared memory. Every >PostgreSQL process collects statistics > locally >then updates the shared data at appropriate intervals. When a server, >including a physical replica, shuts

Re: How about a psql backslash command to show GUCs?

2022-04-09 Thread David G. Johnston
On Sat, Apr 9, 2022 at 9:27 AM Tom Lane wrote: > "Jonathan S. Katz" writes: > > -1, at least for the moment. Sometimes a user doesn't know what they're > > looking for coupled with being unaware of what the default value is. If > > a setting is set to a default value and that value is the

Re: shared-memory based stats collector - v70

2022-04-09 Thread David G. Johnston
On Sat, Apr 9, 2022 at 12:07 PM Andres Freund wrote: > > > ... specific counters. In particular, replay will not increment > > pg_stat_database or pg_stat_all_tables columns, and the startup process > > will not report reads and writes for the pg_statio views. > > > > It would helpful to give

Add version and data directory to initdb output

2022-04-15 Thread David G. Johnston
Hackers, initdb is already pretty chatty, and the version of the cluster being installed seems useful to include as well. The data directory is probably less so - though I am thinking that the absolute path would be useful to report, especially when a relative path is specified (I didn't figure

Re: Improving the "Routine Vacuuming" docs

2022-04-12 Thread David G. Johnston
On Tue, Apr 12, 2022 at 2:53 PM Peter Geoghegan wrote: > Recent work on VACUUM and relfrozenxid advancement required that I > update the maintenance.sgml VACUUM documentation ("Routine > Vacuuming"). It was tricky to keep things current, due in part to > certain structural problems. Many of

Re: Improving the "Routine Vacuuming" docs

2022-04-12 Thread David G. Johnston
On Tue, Apr 12, 2022 at 5:22 PM Peter Geoghegan wrote: > I just don't think that you need to make it any more complicated than > this: physical XID values are only meaningful when compared to other > XIDs from the same cluster. The system needs to make sure that no two > XIDs can ever be more

Re: information_schema and not-null constraints

2023-09-05 Thread David G. Johnston
On Tue, Sep 5, 2023 at 2:50 PM Vik Fearing wrote: > On 9/5/23 19:15, Alvaro Herrera wrote: > > On 2023-Sep-05, Alvaro Herrera wrote: > > > > Looking now at what to do for CHECK_CONSTRAINTS with domain constraints, > > I admit I'm completely confused about what this view is supposed to > > show.

Re: How to add a new pg oid?

2023-09-05 Thread David G. Johnston
OIDs don't exist independently of the data they are associated with. Give more context if you want a better answer. Or just go look at the source code commits for when the last time something needing an OID got added to the core catalog. David J.

Re: How to add a new pg oid?

2023-09-05 Thread David G. Johnston
On Tue, Sep 5, 2023, 11:17 jacktby jacktby wrote: > > > 2023年9月5日 22:29,jacktby jacktby 写道: > > > I’m trying to add a new data type for my pg. How to do that? Can you give > me more details or an example > Use create type and let the system deal with it. Otherwise, no, I don't have that

Re: Can a role have indirect ADMIN OPTION on another role?

2023-09-06 Thread David G. Johnston
On Wed, Sep 6, 2023 at 1:55 PM Ashutosh Sharma wrote: > But what if roleB doesn't want to give roleA access to > the certain objects it owns. Not doable - roleA can always pretend they are roleB one way or another since roleA made roleB. David J.

Re: Improving Physical Backup/Restore within the Low Level API

2023-10-16 Thread David G. Johnston
On Mon, Oct 16, 2023 at 12:09 PM Laurenz Albe wrote: > I think it won't meet with favor if there are cases that require manual > intervention > for starting the server. That was the main argument for getting rid of > the exclusive > backup API, which had a similar problem. > In the rare case

Re: Improving Physical Backup/Restore within the Low Level API

2023-10-16 Thread David G. Johnston
On Mon, Oct 16, 2023 at 10:26 AM Laurenz Albe wrote: > On Mon, 2023-10-16 at 09:26 -0700, David G. Johnston wrote: > > This email is a first pass at a user-visible design for how our backup > and restore > > process, as enabled by the Low Level API, can be modified to make

<    1   2   3   4   5   6   7   8   9   10   >