Re: Addition of authenticated ID to pg_stat_activity

On Tue, Apr 27, 2021 at 09:59:18AM +0900, Michael Paquier wrote: > On Mon, Apr 26, 2021 at 03:21:46PM -0400, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > >> I'm getting a bit worried about the incremental increase in > >> pg_stat_activity width - it's probably by far the

Re: Remove post-increment in function quote_identifier of pg_upgrade

On Thu, Apr 29, 2021 at 06:35:28PM +0530, Vaibhav Dalvi wrote: > Hi, > > The function quote_identifier has extra post-increment operation as > highlighted below, > > char * > quote_identifier(const char *s) > { >char *result = pg_malloc(strlen(s) * 2 + 3); >char *r = result; > >

Re: Patch to allow pg_filedump to support reading of pg_filenode.map

I think you should be able to avoid crashing if passed a non-relmapper file. Make sure not to loop over more mappings than exist in the relmapper file of the given size. I guess you should warn if the number of mappings is too large for the file's size. And then "cap" the number of mappings to

Re: Remove redundant variable from transformCreateStmt

On Thu, Apr 29, 2021 at 02:39:42PM -0400, Alvaro Herrera wrote: > On 2021-Apr-29, Tom Lane wrote: > > Alvaro Herrera writes: > > > I'd do it like this. Note I removed an if/else block in addition to > > > your changes. > > > > > I couldn't convince myself that this is worth pushing though;

Re: pg_upgrade test for binary compatibility of core data types

could do about contrib types. Maybe we should > figure that covering core types is already a step forward, and be > happy with getting that done. Right .. this is meant to at least handle the lowest hanging fruit. -- Justin >From 79bed0997a1c720f103100697bdaa0cb1ee1261d Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date

Re: New assertion failed in lazy_scan_heap() on mereswine

On Sat, May 01, 2021 at 04:43:21PM +1200, Thomas Munro wrote: > TRAP: FailedAssertion("!all_visible_according_to_vm || > prunestate.all_visible", File: > "/home/pgsql/build-farm/buildroot/HEAD/pgsql.build/../pgsql/src/backend/access/heap/vacuumlazy.c", > Line: 1347, PID: 16926) > 2021-04-30

Re: TRUNCATE on foreign table

On Thu, Apr 22, 2021 at 03:36:25PM +0900, Fujii Masao wrote: > diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml > index 553524553b..69aa66e73e 100644 > --- a/doc/src/sgml/fdwhandler.sgml > +++ b/doc/src/sgml/fdwhandler.sgml > @@ -1076,27 +1076,25 @@ ExecForeignTruncate(List

Re: truncating timestamps on arbitrary intervals

On Fri, Apr 09, 2021 at 10:02:47PM +0200, Peter Eisentraut wrote: > On 30.03.21 18:50, John Naylor wrote: > > On Sat, Mar 27, 2021 at 1:06 PM Justin Pryzby wrote: > > > > > > The current docs seem to be missing a "synopsis", like > > > > > &

Re: TRUNCATE on foreign table

On Thu, Apr 22, 2021 at 05:09:02PM +0530, Bharath Rupireddy wrote: > > should it say "specified" instead of requested ? > > Or should it say "requested the RESTART IDENTITY behavior" ? > > > > Also, I think it should say "..otherwise, the CONTINUE IDENTITY behavior was > > requested". > > The

[PATCH] Remove extraneous whitespace in tags: > foo< and >bar

More fixes like the one Peter committed as 9bd563aa9. I eyeballed the HTML to make sure this looks right. >From a8b782cde7c5d6eef1e3876636feb652bc5f3711 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 22 Apr 2021 21:10:49 -0500 Subject: [PATCH] Remove extraneous whitespace in tags

Re: [PATCH] force_parallel_mode and GUC categories

On Wed, Apr 14, 2021 at 03:57:21PM +0900, Michael Paquier wrote: > On Tue, Apr 13, 2021 at 07:31:39AM -0500, Justin Pryzby wrote: > > Good point. > > Thanks. I have used the wording that Tom has proposed upthread, added > one GUC_NOT_IN_SAMPLE that you f

Re: pgsql: autovacuum: handle analyze for partitioned tables

On Thu, Apr 22, 2021 at 12:43:46PM -0500, Justin Pryzby wrote: > Maybe the behavior should be documented, though. Actually, I thought the > pre-existing (non)behavior of autoanalyze would've been documented, and we'd > now update that. All I can find is this: > > https://www.post

Re: [PATCH] force_parallel_mode and GUC categories

On Sat, Apr 24, 2021 at 10:50:21AM +0900, Michael Paquier wrote: > On Fri, Apr 23, 2021 at 01:23:26PM -0500, Justin Pryzby wrote: > > The patch seems to apply cleanly on v12 but cherry-pick needs help for other > > branches... > > FWIW, this did not seem bad enough to me to

strange case of "if ((a & b))"

These look strange to me - the inner parens don't do anything. I wouldn't write it with 2x parens for the same reason I wouldn't write it with 8x parens. diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c index 9f159eb3db..3bbc13c443 100644 ---

Re: Some doubious error messages and comments

ect_agg must be called with a multirange"))); typcache = multirange_get_typcache(fcinfo, mltrngtypoid); commit 8247b4034ed4c68241be9fbdec249bc967ceafd4 Author: Justin Pryzby Date: Tue Apr 27 07:57:50 2021 -0500 Comment typos: extended stats a4d75c86b and 518442c7f diff --git a/src/backend/parser/p

Re: Small issues with CREATE TABLE COMPRESSION

On Tue, Apr 27, 2021 at 03:22:25PM +0900, Michael Paquier wrote: > Hi all, > > I have been looking at and testing the patch set for CREATE TABLE > COMPRESSION, and spotted a couple of things in parallel of some work > done by Jacob (added in CC). > > The behavior around CREATE TABLE AS and

Re: [PATCH] force_parallel_mode and GUC categories

On Fri, Apr 09, 2021 at 10:50:53AM +0900, Michael Paquier wrote: > - {"track_commit_timestamp", PGC_POSTMASTER, REPLICATION, > + {"track_commit_timestamp", PGC_POSTMASTER, REPLICATION_SENDING, > I can get behind this change for clarity where it gets actively used. I'm not sure what

Re: Patch to allow pg_filedump to support reading of pg_filenode.map

This is separate from the postgresql server repo. https://git.postgresql.org/gitweb/?p=pg_filedump.git +#define RELMAPPER_FILEMAGIC 0x592717 +char magic_buffer[8]; ... + if ( (int) magic_buffer & RELMAPPER_FILEMAGIC ) { This is doing bitwise arithmetic on a pointer, which seems badly wrong.

Re: pg_upgrade can result in early wraparound on databases with high transaction load

On Fri, Apr 23, 2021 at 04:42:56PM -0700, Andres Freund wrote: > Hi, > > On 2019-06-15 11:37:59 -0700, Noah Misch wrote: > > On Tue, May 21, 2019 at 03:23:00PM -0700, Peter Geoghegan wrote: > > > On Mon, May 20, 2019 at 3:10 AM Jason Harvey wrote: > > > > This week I upgraded one of my

Re: Testing autovacuum wraparound (including failsafe)

On Fri, Apr 23, 2021 at 01:43:06PM -0700, Andres Freund wrote: > 2) FAILSAFE_MIN_PAGES is 4GB - which seems to make it infeasible to test the >failsafe mode, we can't really create 4GB relations on the BF. While >writing the tests I've lowered this to 4MB... > For 2), I don't really have

Re: PG in container w/ pid namespace is init, process exits cause restart

On Tue, May 04, 2021 at 01:35:50PM -0400, Greg Stark wrote: > Fwiw, I have a suspicion that the right check for being init is > whether `pid == ppid`. pryzbyj@pryzbyj:~$ ps -wwf 1 UIDPID PPID C STIME TTY STAT TIME CMD root 1 0 0 2020 ?Ss10:28 /sbin/init

Re: AlterSubscription_refresh "wrconn" wrong variable?

On Tue, May 04, 2021 at 10:35:02PM -0400, Tom Lane wrote: > Peter Smith writes: > > This patch replaces the global "wrconn" in AlterSubscription_refresh with a > > local variable of the same name, making it consistent with other functions > > in subscriptioncmds.c (e.g. DropSubscription). > >

Re: Extending amcheck to check toast size and compression

+ /* Oversized toasted attributes should never be stored */ + if (toast_pointer.va_rawsize > VARLENA_SIZE_LIMIT) + report_corruption(ctx, + psprintf("toast value %u rawsize %u exceeds limit %u", +

Re: Performance degradation of REFRESH MATERIALIZED VIEW

On Tue, Apr 27, 2021 at 03:43:07PM +0200, Tomas Vondra wrote: > On 4/27/21 7:34 AM, Masahiko Sawada wrote: > > On Tue, Apr 27, 2021 at 8:07 AM Andres Freund wrote: > > > On 2021-04-26 23:59:17 +0200, Tomas Vondra wrote: > > > > On 4/26/21 9:27 PM, Andres Freund wrote: > > > > > On 2021-04-26

Re: TRUNCATE on foreign table

On Thu, Apr 22, 2021 at 07:41:06AM -0700, Zhihong Yu wrote: > > > > + Note that ONLY option specified in > > > > > > add "the" to say: "the ONLY" > > > > +1. > > Since 'the only option' is legitimate English phrase, I think the following > would be clearer: > > Note that the option ONLY ... I

Re: pgsql: autovacuum: handle analyze for partitioned tables

On Wed, Apr 21, 2021 at 07:06:49PM -0400, Alvaro Herrera wrote: > On 2021-Apr-09, Robert Haas wrote: > > Does this need to worry about new partitions getting attached to a > > partitioned table, or old ones getting detached? (Maybe it does > > already, not sure.) > > I was pinged because this is

Re: Add MAIN_RELATION_CLEANUP and SECONDARY_RELATION_CLEANUP options to VACUUM

On Mon, Feb 08, 2021 at 04:35:19PM +0900, Michael Paquier wrote: > On Fri, Jan 29, 2021 at 06:43:44PM +, Bossart, Nathan wrote: > > I changed it to PROCESS_TOAST. > > Thanks. PROCESS_TOAST sounds good to me at the end for the option > name, so let's just go with that. > > > Done. > > While

Re: [HACKERS] Custom compression methods

+* If the called has passed an invalid slot then create a new slot. *caller +* Slot for storing the modified tuple, incase the

Re: [HACKERS] Custom compression methods

This fails make-check world for me, and CFBOT will say the same. Be sure to compile with --enable-tap-tests cd . && TESTDIR='/home/pryzbyj/src/postgres/src/bin/pg_dump' PATH="/home/pryzbyj/src/postgres/tmp_install/usr/local/pgsql/bin:$PATH"

Re: [HACKERS] Custom compression methods

I see the thread got broken somehow (or cfbot thought it did), so I added the new thread, and this is now passing all tests. (I think using the v22 patches). http://cfbot.cputube.org/dilip-kumar.html On Fri, Feb 05, 2021 at 11:07:53AM -0500, Robert Haas wrote: > > Also, I think we may want to

Re: Preserve attstattarget on REINDEX CONCURRENTLY

On Fri, Feb 05, 2021 at 11:17:48AM +0900, Michael Paquier wrote: > Let's copy this data in index_concurrently_swap() instead. The > attached patch does that, and adds a test cheaper than what was > proposed. There is a minor release planned for next week, so I may be > +++

Re: CLUSTER on partitioned index

On Sat, Feb 06, 2021 at 08:45:49AM -0600, Justin Pryzby wrote: > On Mon, Jan 18, 2021 at 12:34:59PM -0600, Justin Pryzby wrote: > > On Sat, Nov 28, 2020 at 08:03:02PM -0600, Justin Pryzby wrote: > > > On Sun, Nov 15, 2020 at 07:53:35PM -0600, Justin Pryzby wrote: > > > &

Re: doc review for v14

Another round of doc fixen. wdiff to follow commit 389c4ac2febe21fd48480a86819d94fd2eb9c1cc Author: Justin Pryzby Date: Wed Feb 10 17:19:51 2021 -0600 doc review for pg_stat_progress_create_index ab0dfc961b6a821f23d9c40c723d11380ce195a6 should backpatch to v13 diff

Re: CREATE INDEX CONCURRENTLY on partitioned index

On Thu, Jan 28, 2021 at 09:51:51PM +0900, Masahiko Sawada wrote: > On Mon, Nov 30, 2020 at 5:22 AM Justin Pryzby wrote: > > On Sat, Oct 31, 2020 at 01:31:17AM -0500, Justin Pryzby wrote: > > > Forking this thread, since the existing CFs have been closed. > > > https:/

Re: create table like: ACCESS METHOD

On Tue, Jan 19, 2021 at 03:03:31PM -0600, Justin Pryzby wrote: > On Wed, Dec 30, 2020 at 12:33:56PM +, Simon Riggs wrote: > > There are no tests for the new functionality, please could you add some? > > Did you look at the most recent patch? > > +CREATE ACCESS METH

Re: [HACKERS] Custom compression methods

On Mon, Mar 22, 2021 at 03:47:58PM -0400, Robert Haas wrote: > On Mon, Mar 22, 2021 at 2:10 PM Tom Lane wrote: > > Robert Haas writes: > > > I think this is significantly cleaner than what we have now, and I > > > also prefer it to your proposal. > > > > +1 in general. However, I suspect that

Re: UPDATE ... SET (single_column) = row_constructor is a bit broken from V10 906bfcad7ba7c

On Mon, Mar 22, 2021 at 02:10:49PM +0530, Mahendra Singh Thalor wrote: > Hi Hackers, > > Commit 906bfcad7ba7c has improved handling for "UPDATE ... SET > (column_list) = row_constructor", but it has broken in some cases where it > was working prior to this commit. > After this commit query “DO

Re: PoC/WIP: Extended statistics on expressions

I got this crash running sqlsmith: #1 0x7f907574b801 in __GI_abort () at abort.c:79 #2 0x5646b95a35f8 in ExceptionalCondition (conditionName=conditionName@entry=0x5646b97411db "bms_num_members(varnos) == 1", errorType=errorType@entry=0x5646b95fa00b "FailedAssertion",

Re: PoC/WIP: Extended statistics on expressions

" 'm' = any(stxkind) > AS mcv_enabled,\n"); > + > + if (pset.sversion >= 13) > + appendPQExpBufferStr(, " stxstattarget\n"); > + else > + app

Re: [POC] Fast COPY FROM command for the table with foreign partitions

On Mon, Mar 22, 2021 at 08:18:56PM -0700, Zhihong Yu wrote: > with data_dest_cb callback. It is used for send text representation of a > tuple to a custom destination. > > send text -> sending text I would say "It is used to send the text representation ..." > struct PgFdwModifyState

Re: PoC/WIP: Extended statistics on expressions

On Wed, Mar 24, 2021 at 09:54:22AM +0100, Tomas Vondra wrote: > Hi Justin, > > Unfortunately the query is incomplete, so I can't quite determine what > went wrong. Can you extract the full query causing the crash, either > from the server log or from a core file? Oh, shoot, I didn't realize it

Re: [HACKERS] Custom compression methods

On Wed, Mar 24, 2021 at 02:24:41PM +0530, Dilip Kumar wrote: > On Wed, Mar 24, 2021 at 1:43 PM Dilip Kumar wrote: > > > create table t1 (col1 text, col2 text); > > > create unique index on t1 ((col1 || col2)); > > > insert into t1 values((select array_agg(md5(g::text))::text from > > >

Re: [HACKERS] Custom compression methods

On Wed, Mar 24, 2021 at 12:24:38PM -0400, Robert Haas wrote: > On Wed, Mar 24, 2021 at 11:42 AM Tom Lane wrote: > > On reflection, though, I wonder if we've made pg_dump do the right > > thing anyway. There is a strong case to be made for the idea that > > when dumping from a pre-14 server, it

Re: PoC/WIP: Extended statistics on expressions

On Wed, Mar 24, 2021 at 05:15:46PM +, Dean Rasheed wrote: > On Wed, 24 Mar 2021 at 16:48, Tomas Vondra > wrote: > > > > As for the changes proposed in the create_statistics, do we really want > > to use univariate / multivariate there? Yes, the terms are correct, but > > I'm not sure how many

Re: [HACKERS] Custom compression methods

On Wed, Mar 24, 2021 at 01:30:26PM -0400, Robert Haas wrote: > On Wed, Mar 24, 2021 at 1:24 PM Justin Pryzby wrote: > > I think it's not specific to pg_upgrade, but any pg_dump |pg_restore. > > > > The analogy with tablespaces is restoring from a cluster where the > &

Re: PoC/WIP: Extended statistics on expressions

On Wed, Mar 24, 2021 at 01:24:46AM -0500, Justin Pryzby wrote: > It seems like you're preferring to use pluralized "statistics" in a lot of > places that sound wrong to me. For example: > > Currently the first statistics wins, which seems silly. > I can write mor

Re: [PATCH] Add extra statistics to explain for Nested Loop

> diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c > index afc45429ba4..723eccca013 100644 > --- a/src/backend/commands/explain.c > +++ b/src/backend/commands/explain.c > @@ -1589,29 +1589,82 @@ ExplainNode(PlanState *planstate, List *ancestors, > double

Re: [HACKERS] Custom compression methods (buildfarm xupgrade)

On Sun, Mar 28, 2021 at 04:48:29PM -0400, Andrew Dunstan wrote: > Nothing is hidden here - the diffs are reported, see for example > > What we're comparing here is

Re: Have I found an interval arithmetic bug?

On Mon, Apr 05, 2021 at 02:01:58PM -0400, Bruce Momjian wrote: > On Mon, Apr 5, 2021 at 11:33:10AM -0500, Justin Pryzby wrote: > > > https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT > > > « …field values can have fractional parts;

Re: Have I found an interval arithmetic bug?

On Fri, Apr 02, 2021 at 10:21:26PM -0400, Bruce Momjian wrote: > I wish I could figure out how to improve it any futher. What is odd is > that I have never seen this reported as a problem before. I plan to > apply this early next week for PG 14. On Fri, Apr 02, 2021 at 01:05:42PM -0700, Bryn

Re: PostgreSQL log query's result size

On Tue, Apr 06, 2021 at 02:03:03PM -0500, Hellmuth Vargas wrote: > Could you tell me if it is possible that as well as the configuration that > the log presents the duration of the delayed queries, it can also present > the size of the result data? especially those who want to return a lot of >

Re: CLUSTER on partitioned index

@cfbot: rebased >From 686cd8fc644f1f86d81d7748b66feddd634c4dc8 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 26 Nov 2020 14:37:08 -0600 Subject: [PATCH v10 1/8] pg_dump: make CLUSTER ON a separate dump object.. ..since it needs to be restored after any child indexes are resto

[PATCH] force_parallel_mode and GUC categories

nder > > > developer options. On Sat, Apr 3, 2021 at 10:41:14AM -0500, Justin Pryzby wrote: > > Because it's there to help DBAs catch errors in functions incorrectly > > marked as > > parallel safe. On Sat, Apr 03, 2021 at 11:43:36AM -0400, Bruce Momjian wrote: > Uh,

Re: ALTER TABLE ADD COLUMN fast default

It reminded me of this thread, but nothing ever came of it. https://www.postgresql.org/message-id/20200328223052.GK20103%40telsasoft.com https://www.postgresql.org/message-id/87pmzaq4gx.fsf%40news-spur.riddles.org.uk

Re: ALTER TABLE ADD COLUMN fast default

On Sun, Apr 04, 2021 at 02:18:34PM -0400, Andrew Dunstan wrote: > On 4/4/21 11:21 AM, Andrew Dunstan wrote: > > On 4/4/21 9:19 AM, Justin Pryzby wrote: > >> It reminded me of this thread, but nothing ever came of it. > >> https://www.postgresql.org/message-id/20200328223

Re: truncating timestamps on arbitrary intervals

On Wed, Mar 24, 2021 at 08:50:59PM +0100, Peter Eisentraut wrote: > On 24.03.21 18:58, John Naylor wrote: > > > As a potential follow-up, should we perhaps add named arguments?  That > > > might make the invocations easier to read, depending on taste. > > > > I think it's quite possible some

Re: [HACKERS] Custom compression methods

On Mon, Mar 22, 2021 at 10:41:33AM -0400, Robert Haas wrote: > trying to explain how TOAST works here, I added a link. It looks, > though, like that documentation also needs to be patched for this > change. I'll look into that, and your remaining patches, next. I added an Opened Item for any

Re: Merging statistics from children instead of re-sampling everything

Thanks for taking a fresh look at this. As you've written it, this can apply to either/both partitioned or inheritence. I imagine when "MERGE" goes away, this should apply only to partitioned tables. (Actually, personally I would advocate to consider applying it to *both*, but I don't think

Re: unconstrained memory growth in long running procedure stored procedure after upgrading 11-12

On Tue, Mar 30, 2021 at 04:17:03PM -0500, Merlin Moncure wrote: > Hello all, > > We just upgraded from postgres 11 to 12.6 and our server is running > out of memory and rebooting about 1-2 times a day.Application > architecture is a single threaded stored procedure, executed with CALL > that

Re: unconstrained memory growth in long running procedure stored procedure after upgrading 11-12

On Tue, Mar 30, 2021 at 04:17:03PM -0500, Merlin Moncure wrote: > Instructions: > 1. run the attached script in psql, pgtask_test.sql. It will create a > database, initialize it, and run the main procedure. dblink must be > available > 2. in another window, run SELECT CreateTaskChain('test',

Re: PoC/WIP: Extended statistics on expressions

On Thu, Mar 25, 2021 at 01:05:37AM +0100, Tomas Vondra wrote: > here's an updated patch. 0001 should address most of the today's review > items regarding comments etc. This is still an issue: postgres=# CREATE STATISTICS xt ON a FROM t JOIN t ON true; ERROR: schema "i" does not exist --

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

On Wed, Dec 23, 2020 at 01:17:10PM -0600, Justin Pryzby wrote: > On Mon, Nov 23, 2020 at 04:14:18PM -0500, Tom Lane wrote: > > * I noticed that you did s/stat/lstat/. That's fine on Unix systems, > > but it won't have any effect on Windows systems (cf bed90759f), > > which

Re: Self-join optimisation

On Thu, Mar 11, 2021 at 03:32:16PM +0100, Matthias van de Meent wrote: > On Thu, 11 Mar 2021 at 15:15, Hywel Carver wrote: > > I asked this question in the Postgres Slack, and was recommended to ask > > here instead. > > > > A few times, I've been in a situation where I want to join a table to

Re: Parallel INSERT (INTO ... SELECT ...)

On Thu, Mar 11, 2021 at 01:01:42PM +, houzj.f...@fujitsu.com wrote: > > I guess to have the finer granularity we'd have to go with > > enable_parallel_insert, > > which then would mean possibly having to later add enable_parallel_update, > > should parallel update have similar potential

Re: [HACKERS] Custom compression methods

On Thu, Mar 11, 2021 at 10:07:30AM +0530, Dilip Kumar wrote: > On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby wrote: > > > > Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was > > calling > > CompareCompressionMethodAndDecompress(). > > Whil

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

On Fri, Mar 12, 2021 at 11:32:27PM +0200, Mark Rofail wrote: > I have retested the patch on a windows build and it passes the regression > tests thanks to Justin's recommendations. Hopefully, it will pass CI too. > > Changelog: > - v7 (compatible with current master 2021-3-12, > commit

Re: Different compression methods for FPI

On Fri, Mar 12, 2021 at 01:45:47AM -0600, Justin Pryzby wrote: > On Sat, Mar 06, 2021 at 12:29:14PM +0500, Andrey Borodin wrote: > > > 1 марта 2021 г., в 10:03, Justin Pryzby написал(а): > > > > Justin, Michael, thanks for comments! > > > > As far as I unde

Re: [HACKERS] Custom compression methods

I'm a minor contributor now to a couple bits of this patch set, but I can answer a couple of these points. On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote: > Comments about 0003: > - why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be > comparable to HIDE_TABLEAM? That

Re: [PATCH]: Allow errors in parameter values to be reported during the BIND phase itself..

On Mon, Mar 15, 2021 at 06:45:49PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > For example: > > $ python3.5 -c "import pg; db=pg.DB(); q = db.query(\"SET > > log_parameter_max_length_on_error=-1;\"); db.prepare('p', 'SELECT > > \$1::smallint');

Re: [PATCH]: Allow errors in parameter values to be reported during the BIND phase itself..

On Mon, Mar 15, 2021 at 08:30:18PM -0400, Tom Lane wrote: > + /* Pop the error callback */ > + error_context_stack = error_context_stack->previous; > + > /* > * Once all parameters have been received, prepare for > printing them >

Re: Different compression methods for FPI

FullTransactionIdIfAny(void); -extern void MarkCurrentTransactionIdLoggedIfAny(void); +extern void MarkCurrentTransactionIdLoggedIfAny(XLogRecPtr upto); +extern void EnsureTopTransactionIdLogged(void); extern bool SubTransactionIsActive(SubTransactionId subxid); extern CommandId GetCurrentComman

Re: Different compression methods for FPI

On Sat, Mar 13, 2021 at 08:48:33PM +0500, Andrey Borodin wrote: > > 13 марта 2021 г., в 06:28, Justin Pryzby написал(а): > > Updated patch with a minor fix to configure.ac to avoid warnings on OSX. > > And 2ndary patches from another thread to allow passing recovery

Re: Parallel INSERT (INTO ... SELECT ...)

On Mon, Mar 15, 2021 at 11:25:26AM +0530, Amit Kapila wrote: > On Fri, Mar 12, 2021 at 3:01 PM houzj.f...@fujitsu.com wrote: > > > > Attaching new version patch with this change. > > Thanks, the patch looks good to me. I have made some minor cosmetic > changes in the attached. I am planning to

Re: libpq compression

On Thu, Mar 18, 2021 at 07:30:09PM +, Daniil Zakhlystov wrote: > The new status of this patch is: Ready for Committer The CF bot is failing , because the last patch sent to the list is from January: | Latest attachment (libpq-compression-31.patch) at 2021-01-12 14:05:22 from Konstantin

Re: libpq compression

On Thu, Mar 18, 2021 at 08:02:32PM -0500, Justin Pryzby wrote: > On Thu, Mar 18, 2021 at 07:30:09PM +, Daniil Zakhlystov wrote: > > The new status of this patch is: Ready for Committer > > The CF bot is failing , because the last patch sent to the list is from > J

Re: [HACKERS] Custom compression methods

On Wed, Mar 10, 2021 at 03:50:48PM -0500, Robert Haas wrote: > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > The pending comment is providing a way to rewrite a table and > > re-compress the data with the current compression method. > > I spent some time poking at this yesterday and ran

Re: pg_upgrade failing for 200+ million Large Objects

432%40sss.pgh.pa.us | Does pg_dump really have sane performance for that situation, or | are we soon going to be fielding requests to make it not be O(N^2) | in the number of listed tables? >From cfc7400bb021659d49170e8b17d067c8e1b9fa33 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue

Re: [HACKERS] Custom compression methods

On Thu, Mar 11, 2021 at 08:17:46AM +0530, Dilip Kumar wrote: > On Thu, Mar 11, 2021 at 2:21 AM Robert Haas wrote: > > > > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > > The pending comment is providing a way to rewrite a table and > > > re-compress the data with the current compression

Re: [HACKERS] Custom compression methods

kov and Justin Pryzby Discussions: https://www.postgresql.org/message-id/20171213151818.75a20...@postgrespro.ru https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ

Re: Asynchronous Append on postgres_fdw nodes.

is_async_capable_path() should probably have a "break" for case T_ForeignPath. little typos: aready sigle givne a event: an event -- Justin

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

On Fri, Mar 19, 2021 at 10:57:37AM -0300, Alvaro Herrera wrote: > > Also, it "fails to avoid" adding duplicate constraints: > > > > Check constraints: > > "c" CHECK (i IS NOT NULL AND i > 1 AND i < 2) > > "cc" CHECK (i IS NOT NULL AND i >= 1 AND i < 2) > > "p1_check" CHECK (true) > >

Re: Different compression methods for FPI

rebased to keep cfbot happy. This will run with default=zlib. On Mon, Mar 15, 2021 at 01:09:18PM -0500, Justin Pryzby wrote: > On Sun, Mar 14, 2021 at 07:31:35PM -0500, Justin Pryzby wrote: > > On Sat, Mar 13, 2021 at 08:48:33PM +0500, Andrey Borodin wrote: > > > > 13 м

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

On Sun, Mar 21, 2021 at 03:01:15PM -0300, Alvaro Herrera wrote: > > But note that it doesn't check if an existing constraint "implies" the new > > constraint - maybe it should. > > Hm, I'm not sure I want to do that, because that means that if I later > have to attach the partition again with the

Re: [HACKERS] Custom compression methods

On Sat, Mar 20, 2021 at 06:20:39PM -0500, Justin Pryzby wrote: > Rebased on HEAD. > 0005 forgot to update compression_1.out. > Included changes to ./configure.ac and some other patches, but not Tomas's, > since it'll make CFBOT get mad as soon as that's pushed. Rebased again

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

On Sun, Mar 21, 2021 at 03:22:00PM -0300, Alvaro Herrera wrote: > On 2021-Mar-21, Justin Pryzby wrote: > > > On Sun, Mar 21, 2021 at 03:01:15PM -0300, Alvaro Herrera wrote: > > > > But note that it doesn't check if an existing constraint "implies" the >

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

On Sun, Mar 21, 2021 at 01:14:20PM -0500, Justin Pryzby wrote: > On Sun, Mar 21, 2021 at 03:01:15PM -0300, Alvaro Herrera wrote: > > > But note that it doesn't check if an existing constraint "implies" the new > > > constraint - maybe it should. > > > >

Re: [HACKERS] Custom compression methods (mac+lz4.h)

On Sun, Mar 21, 2021 at 07:11:50PM -0400, Tom Lane wrote: > I wrote: > > Justin Pryzby writes: > >> On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote: > >>> This seems somewhat repeatable (three identical failures in three > >>> attempts)

Re: [HACKERS] Custom compression methods (mac+lz4.h)

On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote: > This seems somewhat repeatable (three identical failures in three > attempts). Not sure why I did not see it yesterday; but anyway, > there is something wrong with partial detoasting for LZ4. With what version of LZ4 ? -- Justin

Re: [HACKERS] Custom compression methods

On Mon, Mar 22, 2021 at 10:41:33AM -0400, Robert Haas wrote: > On Sun, Mar 21, 2021 at 7:55 PM Justin Pryzby wrote: > > Rebased again. > > Thanks, Justin. I committed 0003 and 0004 together as > 226e2be3876d0bda3dc33d16dfa0bed246b7b74f. I also committed 0001 a

Re: [HACKERS] Custom compression methods

On Mon, Mar 22, 2021 at 11:05:19AM -0400, Robert Haas wrote: > On Mon, Mar 22, 2021 at 10:44 AM Justin Pryzby wrote: > > Thanks. I just realized that if you also push the GUC change, then the docs > > should change from to > > > > doc/src/sgml/config.sgml: &g

Re: [HACKERS] Custom compression methods

On Mon, Mar 22, 2021 at 01:38:36PM -0400, Robert Haas wrote: > On Mon, Mar 22, 2021 at 12:16 PM Robert Haas wrote: > > But, what about giving the default_toast_compression_method GUC an > > assign hook that sets a global variable of type "char" to the > > appropriate value? Then

Re: [HACKERS] Custom compression methods

I sent offlist a couple of times but notice that the latest patch is missing this bit around AC_CHECK_HEADERS, which apparently can sometimes cause warnings on mac. ac_save_CPPFLAGS=$CPPFLAGS CPPFLAGS="$LZ4_CFLAGS $CPPFLAGS" AC_CHECK_HEADERS(lz4/lz4.h, [], [AC_CHECK_HEADERS(lz4.h, [],

Re: [HACKERS] Custom compression methods

On Fri, Mar 19, 2021 at 01:24:39PM -0400, Robert Haas wrote: > On Fri, Mar 19, 2021 at 12:35 PM Justin Pryzby wrote: > > I sent offlist a couple of times but notice that the latest patch is missing > > this bit around AC_CHECK_HEADERS, which apparently can sometimes cause >

Re: pglz compression performance, take two

On Fri, Mar 19, 2021 at 01:29:14PM -0700, Mark Dilger wrote: > Robert Haas just committed Dilip Kumar's LZ4 compression, > bbe0a81db69bd10bd166907c3701492a29aca294. > > Is this pglz compression patch still relevant? How does the LZ4 compression > compare on your hardware? I think it's still

Re: [HACKERS] Custom compression methods

On Fri, Mar 19, 2021 at 10:19:49PM +0100, Tomas Vondra wrote: > On 3/19/21 9:40 PM, Andres Freund wrote: > > On 2021-03-19 17:35:58 -0300, Alvaro Herrera wrote: > >> I find this behavior confusing; I'd rather have configure error out if > >> it can't find the package support I requested, than

Re: [HACKERS] Custom compression methods

;this database's default tblspc". -- Justin >From 58335318a9e72d0dbdf8c2009ba6d195a6cba862 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Fri, 19 Mar 2021 19:12:53 -0500 Subject: [PATCH 1/9] Add docs for default_toast_compression.. bbe0a81db69bd10bd166907c3701492a29aca294 --- doc

Re: [HACKERS] Custom compression methods

On Fri, Mar 19, 2021 at 04:38:03PM -0400, Robert Haas wrote: > On Fri, Mar 19, 2021 at 4:20 PM Tom Lane wrote: > > Nope ... crake's displeased with your assumption that it's OK to > > clutter dumps with COMPRESSION clauses. As am I: that is going to > > be utterly fatal for cross-version

Re: [HACKERS] Custom compression methods

On Sat, Mar 20, 2021 at 01:36:15PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Sat, Mar 20, 2021 at 04:37:53PM +0530, Dilip Kumar wrote: > >> - And, 0008 and 0009, I think my > >> 0001-Fixup-dump-toast-compression-method.patch[1] is doing this in a > &g

Re: [HACKERS] Custom compression methods

On Sat, Mar 20, 2021 at 04:13:47PM +0100, Tomas Vondra wrote: > +++ b/src/backend/access/brin/brin_tuple.c > @@ -213,10 +213,20 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, > BrinMemTuple *tuple, > (atttype->typstorage == TYPSTORAGE_EXTENDED || >

Re: [HACKERS] Custom compression methods

On Sat, Mar 20, 2021 at 04:37:53PM +0530, Dilip Kumar wrote: > - 0006 is fine but not sure what is the advantage over what we have today? The advantage is that it's dozens of lines shorter, and automatically includes a HINT. SET default_toast_compression = 'I do not exist compression';

<    7   8   9   10   11   12   13   14   15   16   >