Re: strange case of "if ((a & b))"

2021-06-24 Thread Justin Pryzby
On Wed, Apr 28, 2021 at 02:40:09PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > These look strange to me - the inner parens don't do anything. > > I wouldn't write it with 2x parens for the same reason I wouldn't write it > > with > > 8x parens. > > A

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-08 Thread Justin Pryzby
On Sun, Jun 06, 2021 at 11:00:38AM -0700, Peter Geoghegan wrote: > On Sun, Jun 6, 2021 at 9:35 AM Justin Pryzby wrote: > > I'll leave the instance running for a little bit before restarting (or > > kill-9) > > in case someone requests more info. > > How about

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-08 Thread Justin Pryzby
On Sun, Jun 06, 2021 at 01:59:10PM -0400, Tom Lane wrote: > Matthias van de Meent writes: > > On Sun, 6 Jun 2021 at 18:35, Justin Pryzby wrote: > >> However, I also found an autovacuum chewing 100% CPU, and it appears the > >> problem is actually because autovac

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-08 Thread Justin Pryzby
On Tue, Jun 08, 2021 at 01:54:41PM +0200, Matthias van de Meent wrote: > On Tue, 8 Jun 2021 at 13:03, Justin Pryzby wrote: > > > > On Sun, Jun 06, 2021 at 11:00:38AM -0700, Peter Geoghegan wrote: > > > On Sun, Jun 6, 2021 at 9:35 AM Justin Pryzby wrote: > > >

Re: Different compression methods for FPI

2021-06-15 Thread Justin Pryzby
On Tue, Jun 15, 2021 at 07:53:26AM +0200, Peter Eisentraut wrote: > On 15.06.21 07:37, Michael Paquier wrote: > > > > Actually, I was just thinking that default yes/no/on/off stuff maybe > > > > should be > > > > defined to mean "lz4" rather than meaning pglz for "backwards > > > >

change logging defaults

2021-06-15 Thread Justin Pryzby
I propose to change some defaults: log_autovacuum_min_duration = 0 log_checkpoints = on log_lock_waits = on (and log_recovery_conflict_waits too?) log_temp_files = 128kB Note that pg_regress does this: | fputs("\n# Configuration added by pg_regress\n\n", pg_conf); |

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-08 Thread Justin Pryzby
On Tue, Jun 08, 2021 at 02:27:14PM +0200, Matthias van de Meent wrote: > Thanks for the information! I created an apparently-complete core file by first doing this: | echo 127 |sudo tee /proc/22591/coredump_filter *and updated wiki:Developer_FAQ to work with huge pages I'm planning to kill the

Re: alter table set TABLE ACCESS METHOD

2021-06-09 Thread Justin Pryzby
On Wed, Jun 09, 2021 at 01:47:18PM +0900, Michael Paquier wrote: > On Tue, Jun 08, 2021 at 05:33:31PM -0700, Jeff Davis wrote: > > New version attached, with the detoasting code removed. Could use > > another round of validation/cleanup in case I missed something during > > the merge. > > This

Re: alter table set TABLE ACCESS METHOD

2021-06-09 Thread Justin Pryzby
On Wed, Jun 09, 2021 at 01:45:52PM -0700, Zhihong Yu wrote: > + /* check if another access method change was already requested > */ > + if (tab->newAccessMethod) > + ereport(ERROR, > + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), > +

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 04:27:47PM +0900, Michael Paquier wrote: > On Tue, May 11, 2021 at 04:42:27PM +0900, Michael Paquier wrote: > > Whatever the solution chosen, the thing I can see we agree on here is > > that we need to do something, at least in the shape of an on/off > > switch to have an

Re: amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 02:22:16PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > Per sqlsmith. > > postgres=# select amvalidate(123); > > ERROR: cache lookup failed for operator class 123 > > postgres=# \errverbose > > ERROR: XX000: cache lookup failed fo

amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Justin Pryzby
Per sqlsmith. postgres=# select amvalidate(123); ERROR: cache lookup failed for operator class 123 postgres=# \errverbose ERROR: XX000: cache lookup failed for operator class 123 LOCATION: amvalidate, amapi.c:125 The usual expectation is that sql callable functions should return null rather

Re: PG 14 release notes, first draft

2021-05-13 Thread Justin Pryzby
On Mon, May 10, 2021 at 09:40:45AM -0500, Justin Pryzby wrote: > Should any of these be included? New SQL-accessible functionality should be included: > 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value. These should be merged: > 756ab29124 Allow pageinspect t

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 05:33:33PM -0400, Alvaro Herrera wrote: > +++ b/doc/src/sgml/maintenance.sgml > @@ -817,6 +817,11 @@ analyze threshold = analyze base threshold + analyze > scale factor * number of tu > > is compared to the total number of tuples inserted, updated, or deleted >

Re: PG 14 release notes, first draft

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote: > These should be merged: > > 756ab29124 Allow pageinspect to inspect GiST indexes (Andrey Borodin, > > Heikki Linnakangas) > > 9e596b65f4 Add "LP_DEAD item?" column to GiST pageinspect functions

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Justin Pryzby
On Fri, May 14, 2021 at 07:50:13PM -0400, Alvaro Herrera wrote: > +++ b/doc/src/sgml/config.sgml > @@ -7643,7 +7643,12 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' > WITH csv; > identifier to be computed. Note that an external module can > alternatively be used if the

Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-05-16 Thread Justin Pryzby
12:41:36PM -0500, Justin Pryzby wrote: > > > Also, I just hit this assertion by cancelling the query with ^C / sigint. > > > But > > > I don't have a reprodcer for it. > > > > > > < 2021-04-17 19:14:23.509 ADT telsasoft >PANIC: LLVMJitContext in use

Re: PG 14 release notes, first draft

2021-05-18 Thread Justin Pryzby
On Tue, May 18, 2021 at 10:07:25AM -0400, Bruce Momjian wrote: > On Tue, May 18, 2021 at 06:28:49PM +0900, Masahiko Sawada wrote: > > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian wrote: > > > > > > I have committed the first draft of the PG 14 release notes. You can > > > see the most current

Re: Different compression methods for FPI

2021-05-18 Thread Justin Pryzby
On Mon, May 17, 2021 at 04:44:11PM +0900, Michael Paquier wrote: > On Sun, Mar 21, 2021 at 02:30:04PM -0500, Justin Pryzby wrote: > > For this patch, this is going to require a bit more in terms of library > linking as the block decompression is done in xlogreader.c, so that's

Re: Inherited UPDATE/DELETE vs async execution

2021-05-09 Thread Justin Pryzby
On Sat, May 08, 2021 at 01:20:51AM +0900, Etsuro Fujita wrote: > I noticed this while working on the > EXPLAIN-ANALYZE-for-async-capable-nodes issue: > > DELETE FROM async_pt; > server closed the connection unexpectedly Confirmed, +Tomas, and added at

Re: PG 14 release notes, first draft

2021-05-10 Thread Justin Pryzby
Thanks for putting it together. I think these two should be merged: | Remove containment operators @ and ~ from contrib modules cube, hstore, intarray, and seg (Justin Pryzby) | Remove deprecated containment operators for built-in geometry data types (Justin Pryzby) | Improve autovacuum's

Re: PG 14 release notes, first draft

2021-05-10 Thread Justin Pryzby
Same as the last couple years, I checked for missing items in the release notes, running something like this. git log --cherry-pick --oneline origin/REL_13_STABLE...origin/master Should any of these be included? f82de5c46b Do COPY FROM encoding conversion/verification in larger chunks.

Re: Re: COPY table_name (single_column) FROM 'unknown.txt' DELIMITER E'\n'

2021-05-05 Thread Justin Pryzby
On Wed, May 05, 2021 at 02:45:41PM -0400, Tom Lane wrote: > > I'm currently using the pg_read_file()-hack in a project, > > and even though it can read files up to 1GB, > > using e.g. regexp_split_to_table() to split on E'\n' > > seems to need 4x as much memory, so it only > > works with files

cache lookup failed for statistics object 123

2021-05-05 Thread Justin Pryzby
f(x) = 11, and the MCV list for the expression shows that 50% of the table has f(x)=11, then the query might decide to *not* use an index scan. -- Justin >From 6dec09300e4ad6cc7977acbfee9db7087420a9b5 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 5 May 2021 04:29:00 -0500 Subj

Re: Printing backtrace of postgres processes

2021-05-05 Thread Justin Pryzby
Here's a cleaned-up copy of the doc text. Send a request to the backend with the specified process ID to log its backtrace. The backtrace will be logged at message level LOG. It will appear in the server log based on the log configuration set (See for more information), but will not be sent to

Re: [v15] ALTER TABLE ... SET ACCESS METHOD

2021-05-05 Thread Justin Pryzby
On Wed, May 05, 2021 at 08:45:50PM -0700, Jeff Davis wrote: > The attached patch implements ALTER TABLE ... SET ACCESS METHOD. > > For simplicity, I used the normal alter table path, ATRewriteTable(), > which does not follow the stricter isolation semantics that VACUUM FULL > follows. If someone

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Justin Pryzby
On Sat, May 08, 2021 at 02:26:44PM +1200, David Rowley wrote: > On Sat, 8 May 2021 at 09:16, Tomas Vondra > wrote: > > On 5/7/21 11:04 PM, David Rowley wrote: > > > Another thought I have is that maybe it would be ok just to move > > > memory accounting debug code so it only runs once in > > >

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-06 Thread Justin Pryzby
On Sun, Jan 10, 2021 at 05:38:50PM -0500, Tom Lane wrote: > I wrote: > > The problems that I see in this area are first that there's no > > real standardization in libpq as to whether to append error messages > > together or just flush preceding messages; and second that no effort > > is made in

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 01:22:27PM -0400, Tom Lane wrote: > I'm curious though why it took this long for anyone to complain. > I'd supposed that people were running sqlsmith against HEAD on > a pretty regular basis. I think it's also becase sqlsmith would need to run against the v14 *client*

Re: Removed extra memory allocations from create_list_bounds

2021-05-17 Thread Justin Pryzby
On Mon, May 17, 2021 at 08:22:25PM +0530, Nitin Jadhav wrote: > I agree and thanks for creating those patches. I am not able to apply > the patch on the latest HEAD. Kindly check and upload the modified > patches. The CFBOT had no issues with the patches, so I suspect an issue on your side.

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-17 Thread Justin Pryzby
On Mon, May 17, 2021 at 01:48:03PM -0700, Andres Freund wrote: > pg_attribute is one of the biggest table in a new cluster, and often the > biggest table in production clusters. Its size is also quite relevant in > memory, due to all the TupleDescs we allocate. > > I just noticed that the new

Re: PG 14 release notes, first draft

2021-05-15 Thread Justin Pryzby
> > > > > On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote: > > > > >> New SQL-accessible functionality should be included: > > > > >>> 8c15a29745 Allow ALTER TYPE to up

Re: PG 14 release notes, first draft

2021-05-12 Thread Justin Pryzby
On Tue, May 11, 2021 at 10:45:04PM -0400, Bruce Momjian wrote: > On Tue, May 11, 2021 at 05:13:21PM -0500, Justin Pryzby wrote: > > On Tue, May 11, 2021 at 10:35:23AM -0400, Bruce Momjian wrote: > > > > | Allow more than the common name (CN) to be matched for client

Re: parallel vacuum - few questions on docs, comments and code

2021-05-11 Thread Justin Pryzby
On Tue, May 11, 2021 at 05:37:50PM +0530, Bharath Rupireddy wrote: > 3) Should the Assert(nindexes > 0); in begin_parallel_vacuum just be > Assert(nindexes > 1); as this function is entered only when indexes > are > 1? I think you're right, at least with the current implementation that

Re: PG 14 release notes, first draft

2021-05-11 Thread Justin Pryzby
nd indexes of a partitioned table (Justin Pryzby, Michael Paquier) It should actually say "all child tables or indexes of a partitioned relation", since you can REINDEX INDEX partitioned_index. | Add bit_xor XOR aggregate function (Alexey Bashtanov) Remove XOR ? | Remove support for

Re: array_cat anycompatible change is breaking xversion upgrade tests

2021-05-20 Thread Justin Pryzby
On Wed, Nov 04, 2020 at 07:43:51PM -0500, Tom Lane wrote: > crake is showing xversion upgrade failures since 9e38c2bb50: > > pg_restore: error: could not execute query: ERROR: function > array_cat(anyarray, anyarray) does not exist > Command was: CREATE AGGREGATE

Re: Removed extra memory allocations from create_list_bounds

2021-05-16 Thread Justin Pryzby
(void *) key); -- 2.17.0 >From 0aeec1e5c9ddf2dea0045ed3ddbf0f7ae09d06a6 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 15 May 2021 15:57:12 -0500 Subject: [PATCH 2/5] allocate the PartitionListValue as a single chunk --- src/backend/partitioning/partbounds.c | 22 ++--

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-08 Thread Justin Pryzby
| You need LZ4, if you want to support the compression of data with this method; see CREATE TABLE. I suggest that should reference guc-default-toast-compression instead of CREATE TABLE, since CREATE TABLE is large and very non-specific. Also, in at least 3 places there's extraneous trailing

Re: PG 14 release notes, first draft

2021-05-11 Thread Justin Pryzby
On Tue, May 11, 2021 at 10:35:23AM -0400, Bruce Momjian wrote: > > | Allow more than the common name (CN) to be matched for client certificate > > authentication (Andrew Dunstan) > > Your description makes it sound like arbitrary attributes can be compared. > > But > > the option just allows

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 01:10:53PM -0700, Jeff Davis wrote: > On Mon, 2021-03-08 at 16:30 +0900, Michael Paquier wrote: > > This toast issue is a kind of interesting one, and it seems rather > > right to rely on toast_build_flattened_tuple() to decompress things > > if > > both table AMs support

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-08 Thread Justin Pryzby
On Sat, May 08, 2021 at 10:13:09PM +0900, Michael Paquier wrote: > + You need LZ4, if you want to support > + the compression of data with this method; see > + . I suggest to change "the compression" to "compression". I would write the whole thing like: | The LZ4 library is needed

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 02:11:31PM -0700, Jeff Davis wrote: > On Thu, 2021-05-06 at 15:23 -0500, Justin Pryzby wrote: > > I think ALTER TABLE SET ACCESS METHOD should move all data off the > > old AM, > > including its toast table. > > Can you explain what you mean,

Re: PG 14 release notes, first draft

2021-05-20 Thread Justin Pryzby
On Thu, May 20, 2021 at 03:44:46PM -0400, Bruce Momjian wrote: > > | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan) > > | This new default better reflects current hardware capabilities. > > Also say: the previous default was 10. > > Uh, we didn't report the new value, so

Re: Incorrect GUC descriptions in docs and postgresql.conf.sample

2021-05-25 Thread Justin Pryzby
On Tue, May 25, 2021 at 08:43:14PM -0500, Justin Pryzby wrote: > Your patch adds documentation about GUCs that can only be set at server > start/config/commandline. Oh: I realized that I read too quickly and misinterpretted what "only be set in the config" means (I know I'm

Re: Incorrect GUC descriptions in docs and postgresql.conf.sample

2021-05-25 Thread Justin Pryzby
Your patch adds documentation about GUCs that can only be set at server start/config/commandline. But it's not true for any of these, which are all HUP/SUSET. Please double check your logic :) src/backend/utils/misc/guc.c: {"autovacuum_work_mem", PGC_SIGHUP, RESOURCES_MEM,

Re: Different compression methods for FPI

2021-05-24 Thread Justin Pryzby
Valid(topfxid)) PG_RETURN_NULL(); + /* the XID is going to be published, make sure it is psersistent */ + EnsureTopTransactionIdLogged(); + PG_RETURN_FULLTRANSACTIONID(topfxid); } diff --git a/src/include/access/xact.h b/src/include/access/xact.h index 134f6862da..593a41

Re: array_cat anycompatible change is breaking xversion upgrade tests

2021-05-25 Thread Justin Pryzby
On Thu, May 20, 2021 at 07:35:10PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Wed, Nov 04, 2020 at 07:43:51PM -0500, Tom Lane wrote: > >> As was discussed in the thread leading up to that commit, modifying the > >> signature of array_cat and friends could b

Re: Removed extra memory allocations from create_list_bounds

2021-05-23 Thread Justin Pryzby
On Sun, May 23, 2021 at 10:40:16PM +0530, Nitin Jadhav wrote: > I have used the same testing procedure as explained in the previous mail. > Please find the timing information of the last 10 creation of partitioned > tables as given below. > Without patch With 0001 and 0002 With all patch ... >

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Justin Pryzby
On Tue, May 25, 2021 at 08:33:47PM -0500, Justin Pryzby wrote: > On Sun, May 23, 2021 at 12:25:10PM -0400, Tom Lane wrote: > > However, the more I looked at that code the less I liked it. > > I think the way that compression selection is handled for indexes,

Re: O_DIRECT on macOS

2021-05-30 Thread Justin Pryzby
On Sun, May 30, 2021 at 04:39:48PM +1200, Thomas Munro wrote: > +BasicOpenFilePermDirect(const char *fileName, int fileFlags, mode_t fileMode, > + bool direct) > ... > +#if !defined(O_DIRECT) && defined(F_NOCACHE) > + /* macOS requires an extra

Re: list of extended statistics on psql (\dX)

2021-05-30 Thread Justin Pryzby
On Wed, Jan 20, 2021 at 11:00:50PM +0100, Tomas Vondra wrote: > Thanks, I've pushed this. I had to tweak the regression tests a bit, for two > reasons: \dX isn't checking schema visibility rules, so accidentally shows stats objects outside of the search path. I noticed after installing the

Re: fdatasync performance problem with large number of DB files

2021-05-29 Thread Justin Pryzby
On Tue, May 25, 2021 at 07:13:59PM -0500, Justin Pryzby wrote: > On Sat, Mar 20, 2021 at 12:16:27PM +1300, Thomas Munro wrote: > > > > + { > > > > + {"recovery_init_sync_method", PGC_POSTMASTER, > > > > ERROR_HANDLING_OPTIONS,

Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-05-28 Thread Justin Pryzby
There's a memory leak affecting JIT expressions, even when inlining, optimization, and tuple deforming are disabled. The server that got OOM after installing PG13.3 (because your patch wasn't applied) still gets OOM even with inline_above_cost=-1, optimize_above_cost=-1, and deforming=off.

Re: What to call an executor node which lazily caches tuples in a hash table? (GUC)

2021-06-02 Thread Justin Pryzby
You started the thread about what to call the node, but what about its GUC? Should it be enable_result_cache instead of enable_resultcache? See also Robert's opinion last year about enable_incrementalsort and "economizing on underscores".

Re: PoC/WIP: Extended statistics on expressions (\d in old client)

2021-06-02 Thread Justin Pryzby
On Fri, Jan 22, 2021 at 02:09:04PM +0100, Tomas Vondra wrote: > On 1/22/21 5:01 AM, Justin Pryzby wrote: > > On Fri, Jan 22, 2021 at 04:49:51AM +0100, Tomas Vondra wrote: > > > > > | Statistics objects: > > > > > | "public"."s2"

Re: PG 14 release notes, first draft

2021-06-04 Thread Justin Pryzby
doc/src/sgml/release-14.sgml- doc/src/sgml/release-14.sgml- doc/src/sgml/release-14.sgml- doc/src/sgml/release-14.sgml- doc/src/sgml/release-14.sgml-Add system view pg_stat_wal doc/src/sgml/release-14.sgml-which reports WAL activity (Masahiro Ikeda)

pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-06 Thread Justin Pryzby
An internal instance was rejecting connections with "too many clients". I found a bunch of processes waiting on a futex and I was going to upgrade the kernel (3.10.0-514) and dismiss the issue. However, I also found an autovacuum chewing 100% CPU, and it appears the problem is actually because

Re: Strangeness with UNIQUE indexes and UTF-8

2021-06-06 Thread Justin Pryzby
On Sun, Jun 06, 2021 at 03:54:48AM -0700, Omar Kilani wrote: > What I sort of don't get is... before we insert anything into these > tables, we always check to see if a value already exists. And Postgres > must be returning no results for some reason. So it goes to insert a > duplicate value which

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-06 Thread Justin Pryzby
On Sun, Jun 06, 2021 at 11:00:38AM -0700, Peter Geoghegan wrote: > On Sun, Jun 6, 2021 at 9:35 AM Justin Pryzby wrote: > > I'll leave the instance running for a little bit before restarting (or > > kill-9) > > in case someone requests more info. > > How about

Re: installcheck failure in indirect_toast with default_toast_compression = lz4

2021-06-06 Thread Justin Pryzby
On Sat, Jun 05, 2021 at 09:20:43AM +0900, Michael Paquier wrote: > As said in $subject, installcheck fails once I set up a server with > default_toast_compression = lz4 in the test indirect_toast. Please > see the attached for the diffs. > > The issue is that the ordering of the tuples returned

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-06 Thread Justin Pryzby
On Sun, Jun 06, 2021 at 07:26:22PM +0200, Matthias van de Meent wrote: > I think it would be helpful for further debugging if we would have the > state of the all tuples on that page (well, the tuple headers with > their transactionids and their line pointers), as that would help with >

Re: when the startup process doesn't

2021-06-06 Thread Justin Pryzby
On Fri, Jun 04, 2021 at 07:49:21PM +0530, Nitin Jadhav wrote: > I have added the proper logs in all of the above scenarios. > > Following is the sample log displayed during server startup when the > time period is set to 10ms. > > 2021-06-04 19:40:06.390 IST [51116] LOG: Syncing data directory,

Re: please update ps display for recovery checkpoint

2021-06-06 Thread Justin Pryzby
t sure if it's okay to assume that's the only caller. Maybe it should check if MyAuxProcType == B_STARTUP. -- Justin >From a143c40fb2bad96d45f5cc3e22f70dd0e2d6b5c6 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 6 Jun 2021 16:28:15 -0500 Subject: [PATCH] Show "syncing data directories&

Re: fdatasync performance problem with large number of DB files

2021-06-04 Thread Justin Pryzby
On Fri, Jun 04, 2021 at 04:24:02PM +0900, Michael Paquier wrote: > On Sat, May 29, 2021 at 02:23:21PM -0500, Justin Pryzby wrote: > > On Tue, May 25, 2021 at 07:13:59PM -0500, Justin Pryzby wrote: > >> On Sat, Mar 20, 2021 at 12:16:27PM +1300,

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-26 Thread Justin Pryzby
On Wed, May 26, 2021 at 11:13:46AM -0400, Tom Lane wrote: > Michael Paquier writes: > > Ah, the parallel with reltablespace and default_tablespace at database > > level is a very good point. It is true that currently the code would > > assign attcompression to a non-zero value once the relation

Re: Speed up pg_checksums in cases where checksum already set

2021-05-26 Thread Justin Pryzby
In one of the checksum patches, there was an understanding that the pages should be written even if the checksum is correct, to handle replicas. >From the v19 patch: https://www.postgresql.org/message-id/F7AFCFCD-8F77-4546-8D42-C7F675A4B680%40yesql.se +* Mark the buffer as dirty

Re: automatic analyze: readahead - add "IO read time" log message

2021-05-25 Thread Justin Pryzby
This patch adds hits/misses/dirtied, but explain says hit/read/dirtied/written. Should it say "read" instead of "misses" ? src/backend/access/heap/vacuumlazy.c: _("buffer usage: %lld hits, %lld misses, %lld dirtied\n"),

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-25 Thread Justin Pryzby
On Sun, May 23, 2021 at 12:25:10PM -0400, Tom Lane wrote: > However, the more I looked at that code the less I liked it. > I think the way that compression selection is handled for indexes, > ie consult default_toast_compression on-the-fly, is *far* saner > than what is currently implemented for

Re: fdatasync performance problem with large number of DB files

2021-05-25 Thread Justin Pryzby
On Sat, Mar 20, 2021 at 12:16:27PM +1300, Thomas Munro wrote: > > > + { > > > + {"recovery_init_sync_method", PGC_POSTMASTER, > > > ERROR_HANDLING_OPTIONS, > > > + gettext_noop("Sets the method for synchronizing the > > > data directory before crash

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-30 Thread Justin Pryzby
On Thu, May 06, 2021 at 01:22:27PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > 52a10224 broke sqlsmith, of all things. > > > It was using errmsg as a test of success, instead of checking if the > > connection > > result wasn't null: > > >

Re: create table like: ACCESS METHOD

2021-06-01 Thread Justin Pryzby
rebased and alphabetized >From c1420e17cc2036d4bf3a6c1f9366bf1cd03bd831 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sun, 15 Nov 2020 16:54:53 -0600 Subject: [PATCH v5] create table (like .. including ACCESS METHOD) --- doc/src/sgml/ref/create_table.sgml|

Re: checking return value from unlink in write_relcache_init_file

2021-06-03 Thread Justin Pryzby
On Thu, Jun 03, 2021 at 03:44:13PM -0700, Zhihong Yu wrote: > Hi, > I was looking at write_relcache_init_file() where an attempt is made to > unlink the tempfilename. > > However, the return value is not checked. > If the tempfilename is not removed (the file exists), I think we should log > a

Re: "debug_invalidate_system_caches_always" is too long

2021-07-04 Thread Justin Pryzby
On Sun, Jul 04, 2021 at 04:27:13PM -0400, Tom Lane wrote: > and the word "always" seems rather confusing --- if it does > something "always", why is there more than one level? So a simple > proposal is to rename it to "debug_invalidate_caches". +1 to remove "always" -- Justin

Re: bugfix: when the blocksize is 32k, the function page_header of pageinspect returns negative numbers.

2021-07-07 Thread Justin Pryzby
On Thu, Jul 08, 2021 at 01:15:12PM +0900, Michael Paquier wrote: > On Thu, Jul 08, 2021 at 09:12:26AM +0530, Bharath Rupireddy wrote: > > +1. int32 makes sense because the maximum allowed block size is 32768 > > and smallint with range -32768 to +32767 can't hold it. Internally, > > lower, upper,

Re: visibility map corruption

2021-07-08 Thread Justin Pryzby
Also, the pg_upgrade status message still seems to be misplaced: In 20210706190612.gm22...@telsasoft.com, Justin Pryzby wrote: > I re-arranged the pg_upgrade output of that patch: it was in the middle of the > two halves: "Setting next transaction ID and epoch for new cluster"

Re: enable_resultcache confusion

2021-07-08 Thread Justin Pryzby
On Thu, Jul 08, 2021 at 12:51:45PM -0400, Bruce Momjian wrote: > Are we going to be forever explaining that enable_resultcache doesn't > cache query results? Do we need a different name? > enable_innerjoin_cache? See also

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-07-02 Thread Justin Pryzby
On Tue, Apr 06, 2021 at 11:01:31AM -0500, Justin Pryzby wrote: > On Wed, Dec 23, 2020 at 01:17:10PM -0600, Justin Pryzby wrote: > > On Mon, Nov 23, 2020 at 04:14:18PM -0500, Tom Lane wrote: > > > * I noticed that you did s/stat/lstat/. That's fine on Unix systems, > >

Re: Signed vs. Unsigned (some)

2021-07-02 Thread Justin Pryzby
On Fri, Jul 02, 2021 at 12:09:23PM +0200, Peter Eisentraut wrote: > On 16.06.21 10:48, Peter Eisentraut wrote: > > On 15.06.21 10:17, Kyotaro Horiguchi wrote: > > > The definitions are not ((type) -1) but ((type) 0x) so > > > actually they might be different if we forget to widen the

Re: PG 14 release notes, first draft

2021-06-25 Thread Justin Pryzby
ent operators @ and ~ for built-in geometric data > types and contrib modules cube, hstore, intarray, and seg (Justin Pryzby) > For example, disregard ^ in its expansion in \1 in (^\d+).*\1. > Add point operators <<| and |>> to be strictly above/below geometry (Emre > Hase

Re: Deadlock risk while inserting directly into partition?

2021-06-25 Thread Justin Pryzby
On Thu, Jun 24, 2021 at 10:27:06AM +1200, David Rowley wrote: > I think the reasons for doing operations directly on partitions are > being reduced with each release. What operations do people really > need to do on partitions now? TRUNCATE is probably one, maybe there's > still a need to CREATE

Re: Different compression methods for FPI

2021-06-26 Thread Justin Pryzby
On Tue, Jun 22, 2021 at 12:53:46PM +0900, Michael Paquier wrote: > > So the patches that you say are unrelated still seem to me to be a > > prerequisite . > > I may be missing something, of course, but I still don't see why > that's necessary? We don't get any test failures on HEAD by switching

Re: track_planning causing performance regression

2021-06-28 Thread Justin Pryzby
On Wed, Apr 21, 2021 at 10:40:07AM -0500, Justin Pryzby wrote: > On Thu, Apr 22, 2021 at 12:13:17AM +0900, Fujii Masao wrote: > > On 2021/04/21 23:53, Justin Pryzby wrote: > > > Or: > > > > > > Enabling this parameter may incur a n

Re: track_planning causing performance regression

2021-06-28 Thread Justin Pryzby
On Tue, Jun 29, 2021 at 10:29:43AM +0800, Julien Rouhaud wrote: > On Tue, Jun 29, 2021 at 10:09 AM Justin Pryzby wrote: > > Is "identical structure" really accurate here? For instance a multi > tenant application could rely on the search_path and only use > unqualified

Re: PG 14 release notes, first draft

2021-06-28 Thread Justin Pryzby
On Mon, Jun 28, 2021 at 09:01:40PM -0400, Bruce Momjian wrote: > On Fri, Jun 25, 2021 at 06:04:56PM -0500, Justin Pryzby wrote: > > > The postgres_fdw supports these type of scans if async_capable is set. > > this type > > remove "The" ? > > New text is:

Re: pg14b2: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", ...

2021-06-28 Thread Justin Pryzby
On Mon, Jun 28, 2021 at 01:42:25PM -0700, Peter Geoghegan wrote: > On Sun, Jun 27, 2021 at 11:08 PM Peter Geoghegan wrote: > > > That said, the relevant table is the active "alarms" table, and it > > > would've > > > gotten plenty of DML with no issue for months running v13. > > > > It might not

Re: enable_resultcache confusion

2021-07-11 Thread Justin Pryzby
On Mon, Jul 12, 2021 at 02:56:58AM +1200, David Rowley wrote: > On Sat, 10 Jul 2021 at 07:30, Robert Haas wrote: > > > > On Fri, Jul 9, 2021 at 11:35 AM David Rowley wrote: > > > I really like that name. > > > > > > I'll wait to see if anyone else wants to voice their opinion before I > > > do

Re: [PATCH] improve the pg_upgrade error message

2021-07-12 Thread Justin Pryzby
On Mon, Jul 12, 2021 at 02:06:31PM +0200, Laurenz Albe wrote: > On Mon, 2021-07-12 at 16:58 +0530, Jeevan Ladhe wrote: > > While looking into one of the pg_upgrade issue, I found it > > challenging to find out the database that has the datallowconn set to > > 'false' that was throwing following

Re: prion failed with ERROR: missing chunk number 0 for toast value 14334 in pg_toast_2619

2021-07-06 Thread Justin Pryzby
On Wed, Jun 30, 2021 at 03:29:41PM +0900, Michael Paquier wrote: > On Tue, May 18, 2021 at 01:04:18PM +0200, Drouvot, Bertrand wrote: > > On 5/17/21 8:56 PM, Andres Freund wrote: > >> On 2021-05-17 20:14:40 +0200, Drouvot, Bertrand wrote: > >>> I was also wondering if: > >>> > >>> * We should

Re: Removed extra memory allocations from create_list_bounds

2021-07-05 Thread Justin Pryzby
Also, if you're going to remove the initializations here, maybe you'd also change i and j to C99 "for" declarations like "for (int i=0, j=0; ...)" - PartitionListValue **all_values = NULL; - ListCell *cell; - int i = 0; - int

Re: Removed extra memory allocations from create_list_bounds

2021-07-05 Thread Justin Pryzby
On Tue, Jul 06, 2021 at 01:48:52AM +1200, David Rowley wrote: > On Wed, 19 May 2021 at 05:28, Nitin Jadhav > wrote: > > I have rebased all the patches on top of > > 'v2_0001-removed_extra_mem_alloc_from_create_list_bounds.patch'. > > Attaching all the patches here. > > I had a look over these

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2021-07-09 Thread Justin Pryzby
On Fri, Jul 09, 2021 at 07:43:02PM -0400, Tom Lane wrote: > Tomas Vondra writes: > > The main question I have is whether this should include procedures. > > I feel a bit uncomfortable about sticking this sort of limited-purpose > selectivity mechanism into pg_dump. I'd rather see a general

Re: unnesting multirange data types

2021-07-10 Thread Justin Pryzby
On Sun, Jul 11, 2021 at 01:00:27AM +0300, Alexander Korotkov wrote: > On Sat, Jul 10, 2021 at 7:34 PM Alvaro Herrera > wrote: > > On 2021-Jun-27, Alexander Korotkov wrote: > > > > > BTW, I found some small inconsistencies in the declaration of > > > multirange operators in the system catalog.

pg14b2: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", ...

2021-06-27 Thread Justin Pryzby
This is crashing repeatedly during insert/update immediately after upgrading an instance to v14, from v13.3. In case it matters, the cluster was originally initdb at 13.2. TRAP: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", Line: 1062, PID: 28580) postgres: telsasoft ts

Re: pg14b2: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", ...

2021-06-27 Thread Justin Pryzby
On Sun, Jun 27, 2021 at 03:08:13PM -0700, Peter Geoghegan wrote: > Can you please amcheck all of the indexes? ts=# SELECT bt_index_check('child.alarms_null_alarm_clear_time_idx'::regclass); ERROR: item order invariant violated for index "alarms_null_alarm_clear_time_idx" DETAIL: Lower index

Re: pg14b2: FailedAssertion("_bt_posting_valid(nposting)", File: "nbtdedup.c", ...

2021-06-27 Thread Justin Pryzby
I've just realized that this VM has a strange storage configuration. It's using LVM thin pools, which I don't use anywhere else. Someone else set this up, and I think I've literally never used pools before. Some time ago, the pool ran out of space, and I ran LVM repair on it. It seems very

Re: PG 14 release notes, first draft

2021-05-19 Thread Justin Pryzby
server variable client_connection_check_interval allows supporting > operating systems, e.g., Linux, to automatically cancel queries by > disconnected clients. say "some operating systems" ? > This can be disabled by turning client option "sslsni" off. "turning of

Re: Removed extra memory allocations from create_list_bounds

2021-05-19 Thread Justin Pryzby
On Tue, May 18, 2021 at 01:49:12PM -0400, Robert Haas wrote: > I see that you have made a theoretical argument for why this should be > good for performance, but it would be better to have some test results > that show that it works out in practice. Sometimes things seem like > they ought to be

Re: Addition of authenticated ID to pg_stat_activity

2021-04-25 Thread Justin Pryzby
On Mon, Apr 26, 2021 at 11:34:16AM +0900, Michael Paquier wrote: > +++ b/doc/src/sgml/config.sgml > @@ -7596,6 +7596,24 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' > WITH csv; > > > > + xreflabel="track_activity_authn_size"> > + track_activity_authn_size >

Re: tab-complete for ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-26 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 04:40:35PM -0400, Alvaro Herrera wrote: > Would anyone oppose me pushing this for tab-completing the new keywords > of ALTER TABLE .. DETACH PARTITION? +1 to apply tab completion for v14 -- Justin

Re: SQL-standard function body

2021-04-26 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 04:04:18PM -0400, Jeff Janes wrote: > This commit break line continuation prompts for unbalanced parentheses in > the psql binary. Skimming through this thread, I don't see that this is > intentional or has been noticed before. > > with psql -X > > Before: > > jjanes=#

<    6   7   8   9   10   11   12   13   14   15   >