Re: CLUSTER on partitioned index

2021-09-23 Thread Justin Pryzby
On Thu, Sep 23, 2021 at 08:18:41PM +0200, Matthias van de Meent wrote: > On Sun, 12 Sept 2021 at 22:10, Justin Pryzby wrote: > > > > On Tue, Jul 20, 2021 at 08:27:02PM -0400, Alvaro Herrera wrote: > > > I have to wonder if there really *is* a use case for CLUSTER in

extended stats on partitioned tables

2021-09-23 Thread Justin Pryzby
extended stats objects are allowed on partitioned tables since v10. https://www.postgresql.org/message-id/flat/CAKJS1f-BmGo410bh5RSPZUvOO0LhmHL2NYmdrC_Jm8pk_FfyCA%40mail.gmail.com 8c5cdb7f4f6e1d6a6104cb58ce4f23453891651b But since 859b3003de they're not populated - pg_statistic_ext(_data) is empty

Re: Release 14 Schedule

2021-09-22 Thread Justin Pryzby
On Wed, Sep 22, 2021 at 12:00:07PM -0400, Jonathan S. Katz wrote: > On 9/22/21 11:15 AM, Magnus Hagander wrote: > > On Wed, Sep 22, 2021 at 5:12 PM Tom Lane wrote: > >> "Jonathan S. Katz" writes: > >>> On 9/19/21 12:32 PM, Justin Pryzby wrote: > >>

Re: Improve logging when using Huge Pages

2021-09-21 Thread Justin Pryzby
On Wed, Sep 22, 2021 at 02:03:11AM +0900, Fujii Masao wrote: > Another idea is to output "Anonymous shared memory was allocated with > huge pages" when it's successfully allocated with huge pages, and to output > "Anonymous shared memory was allocated without huge pages" > when it's successfully

Re: PostgreSQL 14 press release draft

2021-09-20 Thread Justin Pryzby
On Mon, Sep 20, 2021 at 10:19:32PM -0400, Jonathan S. Katz wrote: > PostgreSQL 14 provides a significant throughput boost on workloads that use > many > connections, with some benchmarks showing a 2x speedup. This release continues > on the recent improvements the overall management of B-tree ind

Re: mem context is not reset between extended stats

2021-09-20 Thread Justin Pryzby
On Tue, Sep 21, 2021 at 02:15:45AM +0200, Tomas Vondra wrote: > On 9/15/21 10:09 PM, Justin Pryzby wrote: > > Memory allocation appeared be O(1) WRT the number of statistics objects, > > which > > was not expected to me. This is true in v13 (and probably back to v10). Of

Re: Release 14 Schedule

2021-09-19 Thread Justin Pryzby
when only a few partitions are affected (Amit Langote, Tom Lane) Add SQL-standard SEARCH and CYCLE clauses for common table expressions (Peter Eisentraut) Allow REINDEX to process all child tables or indexes of a partitioned relation (Justin Pryzby, Michael Paquier) BTW I wondered if this should b

Re: PG 14 release notes, first draft

2021-09-18 Thread Justin Pryzby
On Sat, Sep 18, 2021 at 05:15:39PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I think the release notes for the autovacuum item (which was first reverted > > and > > then partially un-reverted) should say something like "Partitioned tables > > are > &

Re: PG 14 release notes, first draft

2021-09-18 Thread Justin Pryzby
I think the release notes for the autovacuum item (which was first reverted and then partially un-reverted) should say something like "Partitioned tables are now included in pg_stat_all_tables": | e1efc5b465 Keep stats up to date for partitioned tables Remove some internal question/marks: ACCURATE

Re: Estimating HugePages Requirements?

2021-09-16 Thread Justin Pryzby
+ * and the hugepage-related mmap flags to use into *mmap_flags. If huge pages + * is not supported, *hugepagesize and *mmap_flags will be set to 0. nitpick: *are* not supported, as you say elsewhere. + gettext_noop("Shows the number of huge pages needed for the main sha

Re: right join with partitioned table crash

2021-09-15 Thread Justin Pryzby
On Wed, Sep 15, 2021 at 07:53:49PM -0400, Tom Lane wrote: > Jaime Casanova writes: > > Here's another crash caught by sqlsmith. > > Fun. Looks like it fails back to v12, but not in v11, > so it's some optimization we added in v12 that's at fault. It seems to be a regression (?) in 12.6 (2021-02

mem context is not reset between extended stats

2021-09-15 Thread Justin Pryzby
kB max resident size >From 6ae4d059f2ed8baf2af92ec0847458055147383c Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 15 Sep 2021 14:38:49 -0500 Subject: [PATCH] stx: do not leak memory for each stats obj --- src/backend/statistics/extended_stats.c | 12 +--- 1 file changed, 5 insertions(+), 7 deletions(-) diff -

Re: proposal: possibility to read dumped table's name from file

2021-09-13 Thread Justin Pryzby
On Wed, Jul 28, 2021 at 09:28:17AM +0200, Pavel Stehule wrote: > Here is an updated implementation of filter's file, that implements syntax > proposed by you. Thanks. If there's any traction for this approach. I have some comments for the next revision, > +++ b/doc/src/sgml/ref/pg_dump.sgml > @

document the need to analyze partitioned tables

2021-09-12 Thread Justin Pryzby
could refer to that. Since the autoanalyze patch was reverted, this should be easily applied to backbranches, which is probably most of its value. commit 4ad2c8f6fd8eb26d76b226e68d3fdb8f0658f113 Author: Justin Pryzby Date: Thu Jul 22 16:06:18 2021 -0500 documentation deficiencies for ANALY

Re: brin multi minmax crash for inet value

2021-09-12 Thread Justin Pryzby
On Sun, Sep 12, 2021 at 07:44:47PM -0500, Jaime Casanova wrote: > Hi Tomas, > > Just noted that this query crash the server. Execute it in the > regression database: If I'm not wrong, this is the crash fixed by e1fbe1181 in April. Could you check what HEAD your server is compiled from ? -- Jus

Re: CLUSTER on partitioned index

2021-09-12 Thread Justin Pryzby
creating a non-brin index for a single day's table (even with v13 deduplication and v12 TID tiebreak). As I see it, support for partitioned cluster is orthogonal to an online/concurrent cluster, which is a job for another patch. -- Justin >From d2e7daf9c05cd3c20c60f5e35c0d6b2612d75d1b Mon Sep 17 00

Re: pg_upgrade test for binary compatibility of core data types

2021-09-11 Thread Justin Pryzby
On Fri, Jul 16, 2021 at 06:02:18PM +, Jacob Champion wrote: > On Fri, 2021-04-30 at 13:33 -0500, Justin Pryzby wrote: > > On Sat, Mar 06, 2021 at 03:01:43PM -0500, Tom Lane wrote: > > > v4-0001 mostly teaches test.sh about specific changes that have to be > > > made

Re: incorrect file name in backend_progress.c header

2021-09-10 Thread Justin Pryzby
> For the first list, do you want to include the path to the file for > IDENTIFICATION ? > If so, I can prepare a patch covering the files in that list. Since there's so few exceptions to the "rule", I think they should be fixed for consistency. Here's an awk which finds a few more - including th

Re: incorrect file name in backend_progress.c header

2021-09-10 Thread Justin Pryzby
mon.c src/port/pgcheckdir.c head pgcheckdir.c src/port/pgcheckdir.c There's some function comments wrong too. In case someone want to fix them together. commit 4fba6c5044da43c1fa263125e422e869ae449ae7 Author: Justin Pryzby Date: Sun Sep 5 18:14:39 2021 -0500 Wrong function name in head

Re: Don't clean up LLVM state when exiting in a bad way

2021-09-07 Thread Justin Pryzby
On Tue, Sep 07, 2021 at 12:27:27PM -0700, Andres Freund wrote: > Hi, > > On 2021-08-18 15:00:59 +, Jelte Fennema wrote: > > I ran into some segfaults when using Postgres that was compiled with LLVM > > 7. According to the backtraces these crashes happened during the call to > > llvm_shutdown,

Re: Improve logging when using Huge Pages

2021-09-07 Thread Justin Pryzby
On Tue, Sep 07, 2021 at 07:12:36PM +0900, Fujii Masao wrote: > One big concern about the patch is that log message is always reported > when shared memory fails to be allocated with huge pages enabled > when huge_pages=try. Since huge_pages=try is the default setting, > many users would see this ne

Re: when the startup process doesn't (logging startup delays)

2021-09-07 Thread Justin Pryzby
On Tue, Sep 07, 2021 at 03:07:15PM +0530, Nitin Jadhav wrote: > > Looking over this version, I realized something I (or you) should have > > noticed sooner: you've added the RegisterTimeout call to > > InitPostgres(), but that's for things that are used by all backends, > > and this is only used by

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-09-06 Thread Justin Pryzby
On Mon, Sep 06, 2021 at 09:18:13PM -0300, Alvaro Herrera wrote: > On 2021-Aug-14, Justin Pryzby wrote: > > > I elaborated on your definition and added here. > > https://commitfest.postgresql.org/34/3285/ > > Thanks! This works for me. After looking at it, it seemed

Re: Correct handling of blank/commented lines in PSQL interactive-mode history

2021-09-06 Thread Justin Pryzby
On Mon, Sep 06, 2021 at 07:50:15AM -0700, David G. Johnston wrote: > On Mon, Sep 6, 2021 at 7:13 AM Greg Nancarrow wrote: > > I've attached a patch that corrects the behaviour. > > For the type of lines mentioned, the patch makes the history behave > > more like Bash history. The behavior of bash

Re: Don't clean up LLVM state when exiting in a bad way

2021-09-05 Thread Justin Pryzby
On Wed, Aug 18, 2021 at 03:00:59PM +, Jelte Fennema wrote: > I ran into some segfaults when using Postgres that was compiled with LLVM 7. > According to the backtraces these crashes happened during the call to > llvm_shutdown, during cleanup after another out of memory condition. It seems >

Re: strange case of "if ((a & b))"

2021-09-05 Thread Justin Pryzby
On Wed, Aug 18, 2021 at 11:08:57PM -0400, Tom Lane wrote: > Peter Smith writes: > > On Thu, Aug 19, 2021 at 4:29 AM Justin Pryzby wrote: > >> - state->oneCol = (origTupdesc->natts == 1) ? true : false; > >> + state->oneCol = origTupdesc->natts

Re: when the startup process doesn't (logging startup delays)

2021-09-03 Thread Justin Pryzby
On Fri, Sep 03, 2021 at 01:23:56PM +0530, Nitin Jadhav wrote: > Please find the updated patch attached. Please check CA+TgmoZtbqxaOLdpNkBcDbz=41tWALA8kpH4M=rwtpyhc7-...@mail.gmail.com I agree with Robert that a standby server should not continuously show timing messages for WAL replay. Some doc

Re: pgsql: Set the volatility of the timestamptz version of date_bin() back

2021-09-03 Thread Justin Pryzby
On Fri, Sep 03, 2021 at 01:56:50PM -0400, Alvaro Herrera wrote: > On 2021-Sep-03, John Naylor wrote: > > > On Fri, Sep 3, 2021 at 1:46 PM Alvaro Herrera > > wrote: > > > > > > On 2021-Sep-03, John Naylor wrote: > > > These catversion bumps in branch 14 this late in the cycle seem suspect. > > > D

Re: PoC/WIP: Extended statistics on expressions

2021-09-02 Thread Justin Pryzby
On Wed, Sep 01, 2021 at 06:45:29PM +0200, Tomas Vondra wrote: > However while polishing the second patch, I realized we're allowing > statistics on expressions referencing system attributes. So this fails; > > CREATE STATISTICS s ON ctid, x FROM t; > > but this passes: > > CREATE STATISTICS s ON

Re: PoC/WIP: Extended statistics on expressions

2021-09-01 Thread Justin Pryzby
On Wed, Sep 01, 2021 at 06:45:29PM +0200, Tomas Vondra wrote: > > > Patch 0001 fixes the "double parens" issue discussed elsewhere in this > > > thread, and patch 0002 tweaks CREATE STATISTICS to treat "(a)" as a simple > > > column reference. > > > > 0002 refuses to create expressional stats on a

Re: mark the timestamptz variant of date_bin() as stable

2021-09-01 Thread Justin Pryzby
On Wed, Sep 01, 2021 at 01:26:26PM -0400, John Naylor wrote: > On Wed, Sep 1, 2021 at 5:32 AM Aleksander Alekseev > wrote: > > > > By looking at timestamptz_bin() implementation I don't see why it > > should be STABLE. Its return value depends only on the input values. > > It doesn't look at the s

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Justin Pryzby
On Mon, Aug 30, 2021 at 09:09:20PM +0200, Laurenz Albe wrote: > On Mon, 2021-08-30 at 17:18 +0200, hubert depesz lubaczewski wrote: > > The thing is - I can't close it with pg_terminate_backend(), and I'd > > rather not kill -9, as it will, I think, close all other connections, > > and this is prod

Re: pg_dump, ATTACH, and independently restorable child partitions

2021-08-30 Thread Justin Pryzby
> the data, etc, but not the ATTACH object.) > > On Mon, Jan 11, 2021 at 09:28:18PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > [ v5-0

Re: Write visibility map during CLUSTER/VACUUM FULL

2021-08-29 Thread Justin Pryzby
class WHERE oid='t'::regclass; SELECT COUNT(1), COUNT(1)FILTER(WHERE all_visible='t') allvis, COUNT(1)FILTER(WHERE all_frozen='t') allfrz, COUNT(1)FILTER(WHERE pd_all_visible='t') allvispd FROM pg_visibility('t'); -- Justin >From 4ec10accb276d9e

extended stats objects are the only thing written like "%s"."%s"

2021-08-28 Thread Justin Pryzby
commit a4d75c86bf15220df22de0a92c819ecef9db3849 Author: Tomas Vondra Date: Fri Mar 26 23:22:01 2021 +0100 Extended statistics on expressions This commit added to psql/describe.c: + /* statistics object name (qualified with namespace) */ +

Re: Estimating HugePages Requirements?

2021-08-27 Thread Justin Pryzby
On Sat, Aug 28, 2021 at 11:00:11AM +0900, Michael Paquier wrote: > On Fri, Aug 27, 2021 at 08:16:40PM +, Bossart, Nathan wrote: > > On 8/27/21, 12:39 PM, "Andres Freund" wrote: > >> One thing I wonder is if this wouldn't better be dealt with in a more > >> generic > >> way. While this is the

\dP and \dX use ::regclass without "pg_catalog."

2021-08-27 Thread Justin Pryzby
;%s\"", | "es.stxrelid::regclass) AS \"%s\"", Tom informs me that this is not considered to be interesting as a security patch. -- Justin >From 70eb6d65084104fa54965a349474cda8db6ed90d M

Re: Autovacuum on partitioned table (autoanalyze)

2021-08-25 Thread Justin Pryzby
On Fri, Aug 20, 2021 at 07:55:13AM -0500, Justin Pryzby wrote: > On Tue, Aug 17, 2021 at 06:30:18AM -0500, Justin Pryzby wrote: > > On Mon, Aug 16, 2021 at 05:28:10PM -0500, Justin Pryzby wrote: > > > On Mon, Aug 16, 2021 at 05:42:48PM -0400, Álvaro Herrera wrote: > > &g

Re: [PATCH] document

2021-08-25 Thread Justin Pryzby
On Wed, Aug 25, 2021 at 09:50:13AM -0400, Tom Lane wrote: > Fujii Masao writes: > > When I applied the patch to the master, I found that the table entries for > > those function were added into the table for aclitem functions in the docs. > > I think this is not valid position and needs to be move

Re: PoC/WIP: Extended statistics on expressions

2021-08-24 Thread Justin Pryzby
On Mon, Aug 16, 2021 at 05:41:57PM +0200, Tomas Vondra wrote: > > This still seems odd: > > > > postgres=# CREATE STATISTICS asf ON i FROM t; > > ERROR: extended statistics require at least 2 columns > > postgres=# CREATE STATISTICS asf ON (i) FROM t; > > CREATE STATISTICS > > > > It seems wrong

Re: Tab completion for "create unlogged" a bit too lax?

2021-08-24 Thread Justin Pryzby
On Tue, Aug 24, 2021 at 11:32:14AM +, Daniel Westermann (DWE) wrote: > tab completion for "create unlogged" gives this: > > postgres=# create unlogged > MATERIALIZED VIEW TABLE > > Given that a materialized table can not be unlogged: > > postgres=# create unlogged materialized view mv1

Re: pgstat_send_connstats() introduces unnecessary timestamp and UDP overhead

2021-08-20 Thread Justin Pryzby
On Tue, Aug 17, 2021 at 02:14:20AM -0700, Andres Freund wrote: > Doubling the number of UDP messages in common workloads seems also problematic > enough that it should be addressed for 14. It increases the likelihood of > dropping stats messages on busy systems, which can have downstream impacts.

Re: Autovacuum on partitioned table (autoanalyze)

2021-08-20 Thread Justin Pryzby
On Tue, Aug 17, 2021 at 06:30:18AM -0500, Justin Pryzby wrote: > On Mon, Aug 16, 2021 at 05:28:10PM -0500, Justin Pryzby wrote: > > On Mon, Aug 16, 2021 at 05:42:48PM -0400, Álvaro Herrera wrote: > > > On 2021-Aug-16, Álvaro Herrera wrote: > > > > > > > Here

Re: strange case of "if ((a & b))"

2021-08-18 Thread Justin Pryzby
On Wed, Aug 18, 2021 at 03:15:21PM -0400, Bruce Momjian wrote: > On Wed, Aug 18, 2021 at 01:28:56PM -0500, Justin Pryzby wrote: > > > > 0002 is a bonus patch I found in my typos branch. I will hold onto it > > > > for > > > > later if nobody wants to deal w

Re: strange case of "if ((a & b))"

2021-08-18 Thread Justin Pryzby
On Wed, Aug 18, 2021 at 02:02:22PM -0400, Bruce Momjian wrote: > On Thu, Jun 24, 2021 at 09:31:11PM -0500, Justin Pryzby wrote: > > On Wed, Apr 28, 2021 at 02:40:09PM -0400, Tom Lane wrote: > > > Justin Pryzby writes: > > > > These look strange to me - the

Re: PoC/WIP: Extended statistics on expressions

2021-08-17 Thread Justin Pryzby
> Patch 0001 fixes the "double parens" issue discussed elsewhere in this > thread, and patch 0002 tweaks CREATE STATISTICS to treat "(a)" as a simple > column reference. > From: Tomas Vondra > Date: Mon, 16 Aug 2021 17:19:33 +0200 > Subject: [PATCH 2/2] fix: identify single-attribute references

Re: Autovacuum on partitioned table (autoanalyze)

2021-08-17 Thread Justin Pryzby
On Mon, Aug 16, 2021 at 05:28:10PM -0500, Justin Pryzby wrote: > On Mon, Aug 16, 2021 at 05:42:48PM -0400, Álvaro Herrera wrote: > > On 2021-Aug-16, Álvaro Herrera wrote: > > > > > Here's the reversal patch for the 14 branch. (It applies cleanly to > >

Re: Autovacuum on partitioned table (autoanalyze)

2021-08-16 Thread Justin Pryzby
On Mon, Aug 16, 2021 at 05:42:48PM -0400, Álvaro Herrera wrote: > On 2021-Aug-16, Álvaro Herrera wrote: > > > Here's the reversal patch for the 14 branch. (It applies cleanly to > > master, but the unused member of PgStat_StatTabEntry needs to be > > removed and catversion bumped). > > I have pu

Re: PoC/WIP: Extended statistics on expressions

2021-08-15 Thread Justin Pryzby
On Mon, Dec 07, 2020 at 03:15:17PM +0100, Tomas Vondra wrote: > > Looking at the current behaviour, there are a couple of things that > > seem a little odd, even though they are understandable. For example, > > the fact that > > > > CREATE STATISTICS s (expressions) ON (expr), col FROM tbl; > >

Re: PoC/WIP: Extended statistics on expressions

2021-08-15 Thread Justin Pryzby
On 1/22/21 5:01 AM, Justin Pryzby wrote: > > In any case, why are there so many parentheses ? On Fri, Jan 22, 2021 at 02:09:04PM +0100, Tomas Vondra wrote: > That's a bug in pg_get_statisticsobj_worker, probably. It shouldn't be > adding extra parentheses, on top of what dep

Re: prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619

2021-08-15 Thread Justin Pryzby
On Sun, May 16, 2021 at 04:23:02PM -0400, Tom Lane wrote: > 1. Fix FullXidRelativeTo to be a little less trusting. It'd > probably be sane to make it return FirstNormalTransactionId > when it'd otherwise produce a wrapped-around FullXid, but is > there any situation where we'd want it to throw an

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-08-14 Thread Justin Pryzby
On Sat, Jul 17, 2021 at 10:45:52AM -0400, Alvaro Herrera wrote: > On 2021-Jul-17, Bharath Rupireddy wrote: > > On Thu, Jul 15, 2021 at 8:17 PM Justin Pryzby wrote: > > > It sounds like something that should be in the glossary, which currently > > > refers > > &

Re: when the startup process doesn't (logging startup delays)

2021-08-14 Thread Justin Pryzby
Should this feature distinguish between crash recovery and archive recovery on a hot standby ? Otherwise the standby will display this all the time. 2021-08-14 16:13:33.139 CDT startup[11741] LOG: redo in progress, elapsed time: 124.42 s, current LSN: 0/EEE2100 If so, I think maybe you'd check

Re: 2021-08-12 release announcement draft

2021-08-11 Thread Justin Pryzby
On Wed, Aug 11, 2021 at 12:32:41PM -0400, Jonathan S. Katz wrote: > * walsenders now show their latest replication commands in `pg_stat_activity`, > instead of just showing the latest SQL command. singular "command" in pg_stat_activity ? > * `pg_settings.pending_restart` now show as true when a

Re: Estimating HugePages Requirements?

2021-08-09 Thread Justin Pryzby
On Thu, Jun 10, 2021 at 07:23:33PM -0500, Justin Pryzby wrote: > On Wed, Jun 09, 2021 at 10:55:08PM -0500, Don Seiler wrote: > > On Wed, Jun 9, 2021, 21:03 P C wrote: > > > > > I agree, its confusing for many and that confusion arises from the fact > > > that y

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Justin Pryzby
On Tue, Aug 03, 2021 at 12:00:47PM -0400, Bruce Momjian wrote: > Patch applied through 9.6. The comment seems to be a leftover from a copy pasto. + /* find hash indexes */ + res = executeQueryOrDie(conn, + "

Re: when the startup process doesn't (logging startup delays)

2021-08-03 Thread Justin Pryzby
Two issues that I saw: The first syncfs message is not output, because it's before InitStartupProgress() is called: 2021-08-03 07:53:02.176 CDT startup[9717] LOG: database system was interrupted; last known up at 2021-08-03 07:52:15 CDT 2021-08-03 07:53:02.733 CDT startup[9717] LOG: data direc

Re: Use WaitLatch for {pre, post}_auth_delay instead of pg_usleep

2021-07-28 Thread Justin Pryzby
On Wed, Jul 28, 2021 at 09:10:35PM +0530, Bharath Rupireddy wrote: > On Wed, Jul 28, 2021 at 8:12 AM Kyotaro Horiguchi > wrote: > > > > - It seems that the additional wait-event is effectively useless for > > most of the processes. Considering that this feature is for debugging > > purpose, it'

Re: needless complexity in StartupXLOG

2021-07-26 Thread Justin Pryzby
On Mon, Jul 26, 2021 at 03:53:20PM -0400, Robert Haas wrote: > Yeah, and there again, it's a lot easier to test if we don't have as > many cases. Now no single change is going to fix that, but the number > of flag variables in xlog.c is simply bonkers. This particular stretch > of code uses 3 of th

Re: when the startup process doesn't (logging startup delays)

2021-07-26 Thread Justin Pryzby
LogStartupProgress(StartupProcessOp operation, const char *path); +extern void LogStartupProgressComplete(StartupProcessOp operation); +extern void LogStartupProgressTimeoutHandler(void); + /* File path names (all relative to $PGDATA) */ #define RECOVERY_SIGNAL_FILE "recovery.signal" #define S

Re: when the startup process doesn't (logging startup delays)

2021-07-25 Thread Justin Pryzby
On Fri, Jul 23, 2021 at 04:09:47PM +0530, Nitin Jadhav wrote: > > I think walkdir() should only call LogStartupProgress(FSYNC_IN_PROGRESS, > > path); > > when action == datadir_fsync_fname. > > I agree and fixed it. I saw that you fixed it by calling InitStartupProgress() after the walkdir() cal

Re: DROP INDEX CONCURRENTLY on partitioned index

2021-07-24 Thread Justin Pryzby
On Wed, Jul 14, 2021 at 09:18:12PM +0530, vignesh C wrote: > The patch does not apply on Head anymore, could you rebase and post a > patch. I'm changing the status to "Waiting for Author". I'm withdrawing this patch at least until the corresponding patch for CIC is progressed. https://commitfest.

Re: Slightly improve initdb --sync-only option's help message

2021-07-22 Thread Justin Pryzby
On Thu, Jul 22, 2021 at 10:32:18PM +, Bossart, Nathan wrote: > On 7/6/21, 7:02 PM, "Gurjeet Singh" wrote: > > I think it would be helpful if the help message was slightly more > > descriptive. Some options: > > > > Used in patch: > > only sync data directory; does not modify any data > >

Re: alter table set TABLE ACCESS METHOD

2021-07-22 Thread Justin Pryzby
On Thu, Jul 22, 2021 at 10:37:12AM +0530, vignesh C wrote: > On Fri, Jul 16, 2021 at 9:19 AM Justin Pryzby wrote: > > > > rebased. > > > > Also, there were two redundant checks for multiple SET ACCESS METHOD > > commands. > > But one of them wasn't hit

Re: when the startup process doesn't (logging startup delays)

2021-07-21 Thread Justin Pryzby
I think walkdir() should only call LogStartupProgress(FSYNC_IN_PROGRESS, path); when action == datadir_fsync_fname. ResetUnloggedRelations() is calling ResetUnloggedRelationsInTablespaceDir("base", op); before calling InitStartupProgress(). This shows StartupXLOG() calling ResetUnloggedRelations(

Re: [PATCH] Automatic HASH and LIST partition creation

2021-07-20 Thread Justin Pryzby
On Tue, Jul 20, 2021 at 02:42:16PM -0400, Robert Haas wrote: > The bigger issue IMHO with on-the-fly > partition creation is avoiding deadlocks in the presence of current > inserters; I submit that without at least some kind of attempt to > avoid deadlocks and spurious errors there, it's not really

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-07-17 Thread Justin Pryzby
On Sat, Jul 17, 2021 at 10:45:52AM -0400, Alvaro Herrera wrote: > On 2021-Jul-17, Bharath Rupireddy wrote: > > > On Thu, Jul 15, 2021 at 8:17 PM Justin Pryzby wrote: > > > > > > On Fri, Jul 09, 2021 at 09:24:19PM +0530, Bharath Rupireddy wrote: > > > >

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Justin Pryzby
On Fri, Jul 16, 2021 at 06:01:12PM -0400, Alvaro Herrera wrote: > On 2021-Jul-16, Justin Pryzby wrote: > > CREATE TABLE p(i int) PARTITION BY RANGE(i); > > CREATE TABLE p1 PARTITION OF p FOR VALUES FROM (1)TO(2); > > CREATE FUNCTION foo() returns trigger LANGUAGE p

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Justin Pryzby
On Fri, Jul 16, 2021 at 02:15:26PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I think there's still an issue that comments on child triggers aren't > > preserved, right ? > > Do we care? That seems like messing with a system-internal object. > In genera

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Justin Pryzby
Thanks for handling this. I think there's still an issue that comments on child triggers aren't preserved, right ? -- Justin

Re: alter table set TABLE ACCESS METHOD

2021-07-15 Thread Justin Pryzby
METHOD Adds support for changing the access method of a table with a rewrite. Author: Justin Pryzby, Jeff Davis --- doc/src/sgml/ref/alter_table.sgml | 20 src/backend/commands/cluster.c | 21 +--- src/backend/commands/matview.c | 5 +- src/backend/command

Re: [PATCH] psql: \dn+ to show size of each schema (and \dA+ for AMs)

2021-07-15 Thread Justin Pryzby
| 0 bytes regress_schema_2 | pryzbyj ||| 0 bytes testxmlschema | pryzbyj ||| 24 kB (10 rows) -- Justin >From d7f5446c0aa1a12a512482d2ed121eb44f6e2f59 Mon Sep 17 00:00:00 2001 From: Justin Pryzby

Re: [PATCH] psql: \dn+ to show size of each schema..

2021-07-15 Thread Justin Pryzby
On Wed, Jul 14, 2021 at 02:05:29PM +0900, Ian Lawrence Barwick wrote: > 2021年7月14日(水) 12:07 Justin Pryzby : > > > > \db+ and \l+ show sizes of tablespaces and databases, so I was surprised in > > the > > past that \dn+ didn't show sizes of schemas. I would find t

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-07-15 Thread Justin Pryzby
On Fri, Jul 09, 2021 at 09:24:19PM +0530, Bharath Rupireddy wrote: > I've always had a hard time distinguishing various types of > processes/terms used in postgres. I look at the source code every time > to understand them, yet I don't feel satisfied with my understanding. > I request any hacker (h

Re: Avoid repeated PQfnumber() in pg_dump

2021-07-14 Thread Justin Pryzby
On Wed, Jul 14, 2021 at 08:54:32AM +, houzj.f...@fujitsu.com wrote: > Since PQfnumber() is not a cheap function, I think we'd better invoke > PQfnumber() out of the loop like the attatched patch. +1 Please add to the next CF -- Justin

[PATCH] psql: \dn+ to show size of each schema..

2021-07-13 Thread Justin Pryzby
| =UC/postgres | | >From c2d68eb54f785c759253d4100460aa1af9cbc676 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 13 Jul 2021 21:25:48 -0500 Subject: [PATCH] psql: \dn+ to show size of each schema.. See also: 358a897fa, 528ac10c7 --- src/bin/psql/describe.c | 5 +

Re: unnesting multirange data types

2021-07-13 Thread Justin Pryzby
On Tue, Jul 13, 2021 at 03:11:16PM +0300, Alexander Korotkov wrote: > On Sun, Jul 11, 2021 at 1:20 AM Justin Pryzby wrote: > > On Sun, Jul 11, 2021 at 01:00:27AM +0300, Alexander Korotkov wrote: > > > On Sat, Jul 10, 2021 at 7:34 PM Alvaro Herrera > > > wrote: > &

Re: [PATCH] improve the pg_upgrade error message

2021-07-12 Thread Justin Pryzby
On Mon, Jul 12, 2021 at 02:06:31PM +0200, Laurenz Albe wrote: > On Mon, 2021-07-12 at 16:58 +0530, Jeevan Ladhe wrote: > > While looking into one of the pg_upgrade issue, I found it > > challenging to find out the database that has the datallowconn set to > > 'false' that was throwing following err

Re: enable_resultcache confusion

2021-07-11 Thread Justin Pryzby
On Mon, Jul 12, 2021 at 02:56:58AM +1200, David Rowley wrote: > On Sat, 10 Jul 2021 at 07:30, Robert Haas wrote: > > > > On Fri, Jul 9, 2021 at 11:35 AM David Rowley wrote: > > > I really like that name. > > > > > > I'll wait to see if anyone else wants to voice their opinion before I > > > do an

Re: unnesting multirange data types

2021-07-10 Thread Justin Pryzby
On Sun, Jul 11, 2021 at 01:00:27AM +0300, Alexander Korotkov wrote: > On Sat, Jul 10, 2021 at 7:34 PM Alvaro Herrera > wrote: > > On 2021-Jun-27, Alexander Korotkov wrote: > > > > > BTW, I found some small inconsistencies in the declaration of > > > multirange operators in the system catalog. No

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-07-09 Thread Justin Pryzby
On Fri, Jul 09, 2021 at 07:43:02PM -0400, Tom Lane wrote: > Tomas Vondra writes: > > The main question I have is whether this should include procedures. > > I feel a bit uncomfortable about sticking this sort of limited-purpose > selectivity mechanism into pg_dump. I'd rather see a general filte

Re: enable_resultcache confusion

2021-07-08 Thread Justin Pryzby
On Thu, Jul 08, 2021 at 12:51:45PM -0400, Bruce Momjian wrote: > Are we going to be forever explaining that enable_resultcache doesn't > cache query results? Do we need a different name? > enable_innerjoin_cache? See also https://www.postgresql.org/message-id/caaphdvos7z90hyiux3kcfe0q_4wywfwfab

Re: visibility map corruption

2021-07-08 Thread Justin Pryzby
Also, the pg_upgrade status message still seems to be misplaced: In 20210706190612.gm22...@telsasoft.com, Justin Pryzby wrote: > I re-arranged the pg_upgrade output of that patch: it was in the middle of the > two halves: "Setting next transaction ID and epoch for new cluster"

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-07 Thread Justin Pryzby
On Thu, Jul 08, 2021 at 01:15:12PM +0900, Michael Paquier wrote: > On Thu, Jul 08, 2021 at 09:12:26AM +0530, Bharath Rupireddy wrote: > > +1. int32 makes sense because the maximum allowed block size is 32768 > > and smallint with range -32768 to +32767 can't hold it. Internally, > > lower, upper, s

Re: prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619

2021-07-06 Thread Justin Pryzby
On Wed, Jun 30, 2021 at 03:29:41PM +0900, Michael Paquier wrote: > On Tue, May 18, 2021 at 01:04:18PM +0200, Drouvot, Bertrand wrote: > > On 5/17/21 8:56 PM, Andres Freund wrote: > >> On 2021-05-17 20:14:40 +0200, Drouvot, Bertrand wrote: > >>> I was also wondering if: > >>> > >>> * We should ke

Re: Removed extra memory allocations from create_list_bounds

2021-07-05 Thread Justin Pryzby
Also, if you're going to remove the initializations here, maybe you'd also change i and j to C99 "for" declarations like "for (int i=0, j=0; ...)" - PartitionListValue **all_values = NULL; - ListCell *cell; - int i = 0; - int ndatum

Re: Removed extra memory allocations from create_list_bounds

2021-07-05 Thread Justin Pryzby
On Tue, Jul 06, 2021 at 01:48:52AM +1200, David Rowley wrote: > On Wed, 19 May 2021 at 05:28, Nitin Jadhav > wrote: > > I have rebased all the patches on top of > > 'v2_0001-removed_extra_mem_alloc_from_create_list_bounds.patch'. > > Attaching all the patches here. > > I had a look over these an

Re: "debug_invalidate_system_caches_always" is too long

2021-07-04 Thread Justin Pryzby
On Sun, Jul 04, 2021 at 04:27:13PM -0400, Tom Lane wrote: > and the word "always" seems rather confusing --- if it does > something "always", why is there more than one level? So a simple > proposal is to rename it to "debug_invalidate_caches". +1 to remove "always" -- Justin

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-07-02 Thread Justin Pryzby
On Tue, Apr 06, 2021 at 11:01:31AM -0500, Justin Pryzby wrote: > On Wed, Dec 23, 2020 at 01:17:10PM -0600, Justin Pryzby wrote: > > On Mon, Nov 23, 2020 at 04:14:18PM -0500, Tom Lane wrote: > > > * I noticed that you did s/stat/lstat/. That's fine on Unix systems, >

Re: Signed vs. Unsigned (some)

2021-07-02 Thread Justin Pryzby
On Fri, Jul 02, 2021 at 12:09:23PM +0200, Peter Eisentraut wrote: > On 16.06.21 10:48, Peter Eisentraut wrote: > > On 15.06.21 10:17, Kyotaro Horiguchi wrote: > > > The definitions are not ((type) -1) but ((type) 0x) so > > > actually they might be different if we forget to widen the consta

Re: track_planning causing performance regression

2021-06-28 Thread Justin Pryzby
On Tue, Jun 29, 2021 at 10:29:43AM +0800, Julien Rouhaud wrote: > On Tue, Jun 29, 2021 at 10:09 AM Justin Pryzby wrote: > > Is "identical structure" really accurate here? For instance a multi > tenant application could rely on the search_path and only use > unqualified

Re: PG 14 release notes, first draft

2021-06-28 Thread Justin Pryzby
On Mon, Jun 28, 2021 at 09:01:40PM -0400, Bruce Momjian wrote: > On Fri, Jun 25, 2021 at 06:04:56PM -0500, Justin Pryzby wrote: > > > The postgres_fdw supports these type of scans if async_capable is set. > > this type > > remove "The" ? > > New text is:

Re: track_planning causing performance regression

2021-06-28 Thread Justin Pryzby
On Wed, Apr 21, 2021 at 10:40:07AM -0500, Justin Pryzby wrote: > On Thu, Apr 22, 2021 at 12:13:17AM +0900, Fujii Masao wrote: > > On 2021/04/21 23:53, Justin Pryzby wrote: > > > Or: > > > > > > Enabling this parameter may incur a n

Re: pg14b2: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", ...

2021-06-28 Thread Justin Pryzby
On Mon, Jun 28, 2021 at 01:42:25PM -0700, Peter Geoghegan wrote: > On Sun, Jun 27, 2021 at 11:08 PM Peter Geoghegan wrote: > > > That said, the relevant table is the active "alarms" table, and it > > > would've > > > gotten plenty of DML with no issue for months running v13. > > > > It might not

Re: pg14b2: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", ...

2021-06-27 Thread Justin Pryzby
I've just realized that this VM has a strange storage configuration. It's using LVM thin pools, which I don't use anywhere else. Someone else set this up, and I think I've literally never used pools before. Some time ago, the pool ran out of space, and I ran LVM repair on it. It seems very possibl

Re: pg14b2: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", ...

2021-06-27 Thread Justin Pryzby
On Sun, Jun 27, 2021 at 03:08:13PM -0700, Peter Geoghegan wrote: > Can you please amcheck all of the indexes? ts=# SELECT bt_index_check('child.alarms_null_alarm_clear_time_idx'::regclass); ERROR: item order invariant violated for index "alarms_null_alarm_clear_time_idx" DETAIL: Lower index tid

pg14b2: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", ...

2021-06-27 Thread Justin Pryzby
This is crashing repeatedly during insert/update immediately after upgrading an instance to v14, from v13.3. In case it matters, the cluster was originally initdb at 13.2. TRAP: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", Line: 1062, PID: 28580) postgres: telsasoft ts 127.

Re: Different compression methods for FPI

2021-06-26 Thread Justin Pryzby
On Tue, Jun 22, 2021 at 12:53:46PM +0900, Michael Paquier wrote: > > So the patches that you say are unrelated still seem to me to be a > > prerequisite . > > I may be missing something, of course, but I still don't see why > that's necessary? We don't get any test failures on HEAD by switching >

Re: Deadlock risk while inserting directly into partition?

2021-06-25 Thread Justin Pryzby
On Thu, Jun 24, 2021 at 10:27:06AM +1200, David Rowley wrote: > I think the reasons for doing operations directly on partitions are > being reduced with each release. What operations do people really > need to do on partitions now? TRUNCATE is probably one, maybe there's > still a need to CREATE I

Re: PG 14 release notes, first draft

2021-06-25 Thread Justin Pryzby
ent operators @ and ~ for built-in geometric data > types and contrib modules cube, hstore, intarray, and seg (Justin Pryzby) > For example, disregard ^ in its expansion in \1 in (^\d+).*\1. > Add point operators <<| and |>> to be strictly above/below geometry (Emre > Hase

<    8   9   10   11   12   13   14   15   16   17   >