Re: how does postgresql handle LOB/CLOB/BLOB column data that dies before the query ends

2023-02-24 Thread Tom Lane
Noel Grandin writes: > On Fri, 24 Feb 2023 at 17:39, Tom Lane wrote: >> Postgres doesn't really do LOB in the same sense that some other DBs >> have, so you'd need to specify what you have in mind in Postgres >> terms to get a useful answer. > So, specifically, the primary problem we have is

Re: Reducing System Allocator Thrashing of ExecutorState to Alleviate FDW-related Performance Degradations

2023-02-24 Thread John Naylor
On Tue, Feb 21, 2023 at 2:46 AM Andres Freund wrote: > On 2023-02-21 08:33:22 +1300, David Rowley wrote: > > I am interested in a bump allocator for tuplesort.c. There it would be > > used in isolation and all the code which would touch pointers > > allocated by the bump allocator would be

Re: how does postgresql handle LOB/CLOB/BLOB column data that dies before the query ends

2023-02-24 Thread Noel Grandin
Thanks for the answers. So, H2, like PostgreSQL, also internally has (a) an MVCC engine and (b) LOBs existing as a on-the-side extra thing. On Fri, 24 Feb 2023 at 17:39, Tom Lane wrote: > Postgres doesn't really do LOB in the same sense that some other DBs > have, so you'd need to specify what

Re: Doc update for pg_stat_statements normalization

2023-02-24 Thread Michael Paquier
On Fri, Feb 24, 2023 at 08:54:00PM +, Imseih (AWS), Sami wrote: > I think the only thing to do here is to call this out in docs with a > suggestion to increase pg_stat_statements.max to reduce the > likelihood. I also attached the suggested doc enhancement as well. Improving the docs about

Re: pg_upgrade and logical replication

2023-02-24 Thread Amit Kapila
On Wed, Feb 22, 2023 at 12:13 PM Julien Rouhaud wrote: > > On Mon, Feb 20, 2023 at 03:07:37PM +0800, Julien Rouhaud wrote: > > On Mon, Feb 20, 2023 at 11:07:42AM +0530, Amit Kapila wrote: > > > > > > I think the current mechanism tries to provide more flexibility to the > > > users. OTOH, in some

Re: Documentation for building with meson

2023-02-24 Thread samay sharma
Hi, On Thu, Dec 1, 2022 at 9:21 AM Andres Freund wrote: > Hi, > > On 2022-12-01 15:58:39 +0100, Peter Eisentraut wrote: > > On 23.11.22 22:24, samay sharma wrote: > > > Thank you. Attaching v7 addressing most of the points below. > > > > I have committed this, after some editing and making some

Re: Add LZ4 compression in pg_dump

2023-02-24 Thread Justin Pryzby
I have some fixes (attached) and questions while polishing the patch for zstd compression. The fixes are small and could be integrated with the patch for zstd, but could be applied independently. - I'm unclear about get_error_func(). That's called in three places from pg_backup_directory.c,

Re: zstd compression for pg_dump

2023-02-24 Thread Justin Pryzby
On Sat, Feb 25, 2023 at 01:44:36PM +0900, Michael Paquier wrote: > On Fri, Feb 24, 2023 at 01:18:40PM -0600, Justin Pryzby wrote: > > This is a draft patch - review is welcome and would help to get this > > ready to be considererd for v16, if desired. > > > > I'm going to add this thread to the

Re: zstd compression for pg_dump

2023-02-24 Thread Michael Paquier
On Fri, Feb 24, 2023 at 01:18:40PM -0600, Justin Pryzby wrote: > This is a draft patch - review is welcome and would help to get this > ready to be considererd for v16, if desired. > > I'm going to add this thread to the old CF entry. > https://commitfest.postgresql.org/31/2888/ Patch 0003 adds

Re: verbose mode for pg_input_error_message?

2023-02-24 Thread Michael Paquier
On Fri, Feb 24, 2023 at 05:36:42PM -0500, Corey Huinker wrote: > Looks good to me, passes make check-world. Thanks for slogging through this. FWIW, I agree that switching pg_input_error_message() to return a row would be nicer in the long-run than just getting an error message because it has the

Re: Proposal: :SQL_EXEC_TIME (like :ROW_COUNT) Variable (psql)

2023-02-24 Thread Kirk Wolak
On Fri, Feb 24, 2023 at 7:09 AM Jim Jones wrote: > On 23.02.23 20:55, Kirk Wolak wrote: > > Everyone, > ... SQL_EXEC_TIME > > I think like ROW_COUNT, it should not change because of internal > > commands. > > So, you guys +1 this thing, give additional comments. When the > > feedback settles,

Re: Proposal: :SQL_EXEC_TIME (like :ROW_COUNT) Variable (psql)

2023-02-24 Thread Kirk Wolak
On Fri, Feb 24, 2023 at 2:11 AM Gurjeet Singh wrote: > On Thu, Feb 23, 2023 at 8:42 PM Kirk Wolak wrote: > > I love that my proposal for %T in the prompt, triggered some great > conversations. > > > > This is not instead of that. That lets me run a query and come back > HOURS later, and

Re: Disable vacuuming to provide data history

2023-02-24 Thread Vik Fearing
On 2/24/23 22:06, Corey Huinker wrote: On Thu, Feb 23, 2023 at 6:04 AM wrote: [1] some implementations don't use null, they use an end-timestamp set to a date implausibly far in the future ( 3999-12-31 for example ), The specification is, "At any point in time, all rows that have their

Re: Move defaults toward ICU in 16?

2023-02-24 Thread Jeff Davis
On Fri, 2023-02-17 at 15:07 -0800, Jeff Davis wrote: > 2. Update the pg_database entry for template0. This has less > potential > for surprise in case people are actually using template0 for a > template. New patches attached. 0001: default autoconf to build with ICU (meson already uses

Re: Disable rdns for Kerberos tests

2023-02-24 Thread Stephen Frost
Greetings, * Heikki Linnakangas (hlinn...@iki.fi) wrote: > On 21/02/2023 01:35, Stephen Frost wrote: > > The name canonicalization support for Kerberos is doing us more harm > > than good in the regression tests, so I propose we disable it. Patch > > attached. > > > > Thoughts? > > Makes

Re: [Proposal] Allow pg_dump to include all child tables with the root table

2023-02-24 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Hi the patch applies fine on current master branch and it works as

Re: verbose mode for pg_input_error_message?

2023-02-24 Thread Corey Huinker
On Thu, Feb 23, 2023 at 4:47 PM Nathan Bossart wrote: > On Thu, Feb 23, 2023 at 11:30:38AM -0800, Nathan Bossart wrote: > > Will post a new version shortly. > > As promised... > > -- > Nathan Bossart > Amazon Web Services: https://aws.amazon.com Looks good to me, passes make check-world.

Re: PG_FREE_IF_COPY extraneous in numeric_cmp?

2023-02-24 Thread Tom Lane
CK Tan writes: > Isn't it true that pfree() will never be called by PG_FREE_IF_COPY? No. You're forgetting the possibility that PG_GETARG_NUMERIC will have to de-toast a toasted input. Granted, numerics are seldom going to be long enough to get compressed or pushed out-of-line; but that's

Re: BRIN indexes vs. SK_SEARCHARRAY (and preprocessing scan keys)

2023-02-24 Thread Heikki Linnakangas
I had a quick look at just the preliminary cleanup patches: 0001-BRIN-bloom-cleanup-20230218.patch Looks good to me 0002-BRIN-minmax-multi-cleanup-20230218.patch Looks good, although it would feel more natural to me to do it the other way round, and define 'matches' as 'bool matches',

Re: Disable vacuuming to provide data history

2023-02-24 Thread Corey Huinker
On Thu, Feb 23, 2023 at 6:04 AM wrote: > Hey, > > It depnends on scenario, but there is many use cases that hack data > change from somebody with admin privileges could be disaster. > That is the place where data history could come with help. Some basic > solution would be trigger which writes

Doc update for pg_stat_statements normalization

2023-02-24 Thread Imseih (AWS), Sami
Replacing constants in pg_stat_statements is on a best effort basis. It is not unlikely that on a busy workload with heavy entry deallocation, the user may observe the query with the constants in pg_stat_statements. From what I can see, this is because the only time an entry is normalized is

Re: Marking options deprecated in help output

2023-02-24 Thread Heikki Linnakangas
On 05/12/2022 11:42, Daniel Gustafsson wrote: In the pg_dump blob terminology thread it was briefly discussed [0] to mark parameters as deprecated in the --help output. The attached is a quick diff to show that that would look like. Personally I think it makes sense, not everyone will read the

Re: Inconsistency in ACL error message

2023-02-24 Thread Joseph Koshakow
On Fri, Feb 24, 2023 at 1:31 PM Nathan Bossart wrote: > You might be interested in > >https://commitfest.postgresql.org/42/4145/ Ah, perfect. In that case ignore my patch! - Joe Koshakow

zstd compression for pg_dump

2023-02-24 Thread Justin Pryzby
This is a draft patch - review is welcome and would help to get this ready to be considererd for v16, if desired. I'm going to add this thread to the old CF entry. https://commitfest.postgresql.org/31/2888/ -- Justin >From 2486417b7c3586e150e806a1fbc3b873c2a4a0f9 Mon Sep 17 00:00:00 2001 From:

Re: PATCH: Using BRIN indexes for sorted output

2023-02-24 Thread Tomas Vondra
On 2/24/23 19:03, Matthias van de Meent wrote: > On Thu, 23 Feb 2023 at 19:48, Tomas Vondra > wrote: >> >> On 2/23/23 17:44, Matthias van de Meent wrote: >>> On Thu, 23 Feb 2023 at 16:22, Tomas Vondra >>> wrote: On 2/23/23 15:19, Matthias van de Meent wrote: > Comments on 0001,

PG_FREE_IF_COPY extraneous in numeric_cmp?

2023-02-24 Thread CK Tan
Hi hackers, I have a question on the code below: Datum numeric_cmp(PG_FUNCTION_ARGS) { Numeric num1 = PG_GETARG_NUMERIC(0); Numeric num2 = PG_GETARG_NUMERIC(1); int result; result = cmp_numerics(num1, num2); PG_FREE_IF_COPY(num1, 0); PG_FREE_IF_COPY(num2, 1);

Re: Inconsistency in ACL error message

2023-02-24 Thread Nathan Bossart
On Fri, Feb 24, 2023 at 12:23:27PM -0500, Joseph Koshakow wrote: > I noticed a very minor inconsistency in some ACL error messages. When > you are try and alter a role, it just says "permission denied": You might be interested in https://commitfest.postgresql.org/42/4145/ -- Nathan

Re: PATCH: Using BRIN indexes for sorted output

2023-02-24 Thread Matthias van de Meent
On Thu, 23 Feb 2023 at 19:48, Tomas Vondra wrote: > > On 2/23/23 17:44, Matthias van de Meent wrote: > > On Thu, 23 Feb 2023 at 16:22, Tomas Vondra > > wrote: > >> > >> On 2/23/23 15:19, Matthias van de Meent wrote: > >>> Comments on 0001, mostly comments and patch design: > > > > One more

Inconsistency in ACL error message

2023-02-24 Thread Joseph Koshakow
Hi all, I noticed a very minor inconsistency in some ACL error messages. When you are try and alter a role, it just says "permission denied": postgres=> ALTER ROLE bar NOCREATEDB; ERROR: permission denied postgres=> ALTER ROLE bar SET search_path TO 'foo'; ERROR: permission denied For

Re: Timeline ID hexadecimal format

2023-02-24 Thread Sébastien Lardière
On 31/01/2023 20:16, Greg Stark wrote: A hint or something just in that case might be enough? It seems to be a -1 ; let's try to improve the documentation, with the attached patch best regards, -- Sébastien diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index

Re: PATCH: Using BRIN indexes for sorted output

2023-02-24 Thread Matthias van de Meent
On Fri, 24 Feb 2023 at 17:04, Tomas Vondra wrote: > > On 2/24/23 16:14, Alvaro Herrera wrote: > > ... if pagesPerRange is not a whole divisor of MaxBlockNumber, I think > > this will neglect the last range in the table. > > > > Why would it? Let's say BlockNumber is uint8, i.e. 255 max. And there

Re: PATCH: Using BRIN indexes for sorted output

2023-02-24 Thread Tomas Vondra
On 2/24/23 16:14, Alvaro Herrera wrote: > On 2023-Feb-24, Tomas Vondra wrote: > >> I guess the easiest fix would be to do the arithmetic in 64 bits. That'd >> eliminate the overflow. > > Yeah, that might be easy to set up. We then don't have to worry about > it until BlockNumber is enlarged

Re: Missing update of all_hasnulls in BRIN opclasses

2023-02-24 Thread Tomas Vondra
On 1/9/23 00:34, Tomas Vondra wrote: > > I've been working on this over the past couple days, trying to polish > and commit it over the weekend - both into master and backbranches. > Sadly, the backpatching part turned out to be a bit more complicated > than I expected, because of the BRIN

Re: how does postgresql handle LOB/CLOB/BLOB column data that dies before the query ends

2023-02-24 Thread Tom Lane
Noel Grandin writes: > Hacker from another open-source DB here (h2database.com). > How does postgresql handle the following situation? > (1) a table containing a LOB column Postgres doesn't really do LOB in the same sense that some other DBs have, so you'd need to specify what you have in mind

Re: Stale references to guc.c in comments/tests

2023-02-24 Thread Tom Lane
Daniel Gustafsson writes: > I happened to notice that there were a few references to guc.c regarding > variables, which with the recent refactoring in 0a20ff54f have become stale. > Attached is a trivial patch to instead point to guc_tables.c. Hmm, I think you may have done an overenthusiastic

Re: PATCH: Using BRIN indexes for sorted output

2023-02-24 Thread Alvaro Herrera
On 2023-Feb-24, Tomas Vondra wrote: > I guess the easiest fix would be to do the arithmetic in 64 bits. That'd > eliminate the overflow. Yeah, that might be easy to set up. We then don't have to worry about it until BlockNumber is enlarged to 64 bits ... but by that time surely we can just grow

Re: pgindent vs. git whitespace check

2023-02-24 Thread Peter Eisentraut
On 22.02.23 15:49, Alvaro Herrera wrote: On 2023-Feb-22, Peter Eisentraut wrote: In the meantime, I suggest we work around this, perhaps by conn = libpqsrv_connect_params(keywords, values, /* expand_dbname = */ false, PG_WAIT_EXTENSION); I

how does postgresql handle LOB/CLOB/BLOB column data that dies before the query ends

2023-02-24 Thread Noel Grandin
Hi Hacker from another open-source DB here (h2database.com). How does postgresql handle the following situation? (1) a table containing a LOB column (2) a query that does ResultSet rs = query("select lob_column from table_foo"); while (rs.next()) { retrieve_lob_data(rs.getLob(1));

Re: buildfarm + meson

2023-02-24 Thread Andrew Dunstan
On 2023-02-23 Th 16:12, Andrew Dunstan wrote: On 2023-02-23 Th 10:58, Andres Freund wrote: On a Windows instance, fairly similar to what's running drongo, I can get a successful build with meson+VS2019, but I'm getting an error in the regression tests, which don't like setting lc_time to

Stale references to guc.c in comments/tests

2023-02-24 Thread Daniel Gustafsson
I happened to notice that there were a few references to guc.c regarding variables, which with the recent refactoring in 0a20ff54f have become stale. Attached is a trivial patch to instead point to guc_tables.c. -- Daniel Gustafsson 0001-Fix-outdated-references-to-guc.c.patch Description:

Re: meson: Non-feature feature options

2023-02-24 Thread Nazir Bilal Yavuz
Hi, On Wed, 22 Feb 2023 at 12:14, Peter Eisentraut wrote: > > On 21.02.23 17:32, Nazir Bilal Yavuz wrote: > >>> I like the second approach, with a 'uuid' feature option. As you wrote > >>> earlier, adding an 'auto' choice to a combo option doesn't work fully > >>> like a > >>> real feature

Re: Proposal: :SQL_EXEC_TIME (like :ROW_COUNT) Variable (psql)

2023-02-24 Thread Jim Jones
On 23.02.23 20:55, Kirk Wolak wrote: Everyone,   I love that my proposal for %T in the prompt, triggered some great conversations.   This is not instead of that.  That lets me run a query and come back HOURS later, and know it finished before 7PM like it was supposed to!   This feature is

Re: dynamic result sets support in extended query protocol

2023-02-24 Thread Peter Eisentraut
On 20.02.23 13:58, Peter Eisentraut wrote: The attached patches are the same as before, rebased over master and split up as described.  I haven't done any significant work on the contents, but I will try to get the 0001 patch into a more polished state soon. I've done a bit of work on this

Re: allow meson to find ICU in non-standard localtion

2023-02-24 Thread Nazir Bilal Yavuz
Hi, Thanks for the patch. On Wed, 22 Feb 2023 at 21:26, Jeff Davis wrote: > > I'm not sure it's the right thing to do though. One downside is that it > doesn't output the version that it finds, it only outputs "YES". - icu = dependency('icu-uc', required: icuopt.enabled()) - icu_i18n =

Re: PATCH: Using BRIN indexes for sorted output

2023-02-24 Thread Tomas Vondra
On 2/24/23 09:39, Alvaro Herrera wrote: > On 2023-Feb-23, Matthias van de Meent wrote: > >>> + for (heapBlk = 0; heapBlk < nblocks; heapBlk += pagesPerRange) >> >> I am not familiar with the frequency of max-sized relations, but this >> would go into an infinite loop for pagesPerRange values

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2023-02-24 Thread Anton A. Melnikov
Hi, Thomas! On 17.02.2023 06:21, Thomas Munro wrote: There are two kinds of atomicity that we rely on for the control file today: * atomicity on power loss (= device property, in case of overwrite filesystems) * atomicity of concurrent reads and writes (= VFS or kernel buffer pool interlocking

Re: TAP output format in pg_regress

2023-02-24 Thread Daniel Gustafsson
Another rebase on top of 337903a16f. Unless there are conflicting reviews, I consider this patch to be done and ready for going in during the next CF. -- Daniel Gustafsson v17-0001-Emit-TAP-compliant-output-from-pg_regress.patch Description: Binary data

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2023-02-24 Thread Corey Huinker
On Thu, Feb 23, 2023 at 2:39 PM Andres Freund wrote: > Hi, > > On 2023-02-23 13:56:56 -0500, Tom Lane wrote: > > Corey Huinker writes: > > > My not-ready-for-16 work on CAST( ... ON DEFAULT ... ) involved making > > > FuncExpr/IoCoerceExpr/ArrayCoerceExpr have a safe_mode flag, and that > > >

Re: Doc updates for MERGE

2023-02-24 Thread Dean Rasheed
On Fri, 24 Feb 2023 at 08:56, Alvaro Herrera wrote: > > Agreed. Your patch looks good to me. > > I was confused for a bit about arch-dev.sgml talking about ModifyTable > when perform.sgml talks about Insert/Update et al; I thought at first > that one or the other was in error, so I checked. It

Re: Add LZ4 compression in pg_dump

2023-02-24 Thread gkokolatos
--- Original Message --- On Friday, February 24th, 2023 at 5:35 AM, Michael Paquier wrote: > > > On Thu, Feb 23, 2023 at 07:51:16PM -0600, Justin Pryzby wrote: > > > On Thu, Feb 23, 2023 at 09:24:46PM +0100, Tomas Vondra wrote: > > > > > I've now pushed 0002 and 0003, after

Re: Doc updates for MERGE

2023-02-24 Thread Alvaro Herrera
On 2023-Feb-24, Dean Rasheed wrote: > Attached is a patch fixing a few doc omissions for MERGE. > > I don't think that it's necessary to update every place that could > possibly apply to MERGE, but there are a few places where we give a > list of commands that may be used in a particular

Re: Should vacuum process config file reload more often

2023-02-24 Thread Pavel Borisov
Hi, Melanie! On Fri, 24 Feb 2023 at 02:08, Melanie Plageman wrote: > > Hi, > > Users may wish to speed up long-running vacuum of a large table by > decreasing autovacuum_vacuum_cost_delay/vacuum_cost_delay, however the > config file is only reloaded between tables (for autovacuum) or after > the

Re: Support logical replication of DDLs

2023-02-24 Thread Masahiko Sawada
On Tue, Feb 21, 2023 at 11:09 AM Zheng Li wrote: > > On Mon, Feb 20, 2023 at 3:23 AM Masahiko Sawada wrote: > > > > On Fri, Feb 17, 2023 at 1:13 PM Zheng Li wrote: > > > > > > > > I've implemented a prototype to allow replicated objects to have the > > > > > same owner from the publisher in > >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-02-24 Thread Masahiko Sawada
On Thu, Feb 23, 2023 at 6:41 PM John Naylor wrote: > > I ran a couple "in situ" tests on server hardware using UUID columns, since > they are common in the real world and have bad correlation to heap order, so > are a challenge for index vacuum. Thank you for the test! > > === test 1, delete

Re: PATCH: Using BRIN indexes for sorted output

2023-02-24 Thread Alvaro Herrera
On 2023-Feb-23, Matthias van de Meent wrote: > > + for (heapBlk = 0; heapBlk < nblocks; heapBlk += pagesPerRange) > > I am not familiar with the frequency of max-sized relations, but this > would go into an infinite loop for pagesPerRange values >1 for > max-sized relations due to BlockNumber