Re: Perform streaming logical transactions by background workers and parallel apply

On Mon, Aug 8, 2022 at 11:41 AM Dilip Kumar wrote: > > On Mon, Aug 8, 2022 at 10:18 AM Dilip Kumar wrote: > > > > > Based on above, we plan to first introduce the patch to perform streaming > > > logical transactions by background workers, and then introduce parallel > > > apply > > > normal

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

On Tue, Aug 9, 2022 at 4:28 PM Nathan Bossart wrote: > On Tue, Aug 09, 2022 at 09:29:16AM +0530, Bharath Rupireddy wrote: > > On Tue, Aug 9, 2022 at 9:20 AM Tom Lane wrote: > >> Hmmm ... I'll grant that ignoring lstat errors altogether isn't great. > >> But should the replacement behavior be

Re: optimize lookups in snapshot [sub]xip arrays

Nathan Bossart writes: > On Tue, Aug 09, 2022 at 09:40:15AM +0530, Bharath Rupireddy wrote: >> Isn't it a good idea to capture the above two points as comments in >> port/pg_lfind.h just to not lose track of it? I know these are present >> in the hackers thread, but having them in the form of

Re: optimize lookups in snapshot [sub]xip arrays

On Tue, Aug 09, 2022 at 09:40:15AM +0530, Bharath Rupireddy wrote: > Isn't it a good idea to capture the above two points as comments in > port/pg_lfind.h just to not lose track of it? I know these are present > in the hackers thread, but having them in the form of comments helps > developers who

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

On Tue, Aug 09, 2022 at 09:29:16AM +0530, Bharath Rupireddy wrote: > On Tue, Aug 9, 2022 at 9:20 AM Tom Lane wrote: >> Hmmm ... I'll grant that ignoring lstat errors altogether isn't great. >> But should the replacement behavior be elog-LOG-and-press-on, >> or

Re: optimize lookups in snapshot [sub]xip arrays

On Tue, Aug 9, 2022 at 4:37 AM Nathan Bossart wrote: > > On Mon, Aug 08, 2022 at 12:56:28PM +0530, Bharath Rupireddy wrote: > > 1) pg_lfind32 - why just uint32? If it's not possible to define > > functions for char, unsigned char, int16, uint16, int32, int64, uint64 > > and so on, can we add a

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

On 2022-08-09 15:10:41 +1200, Thomas Munro wrote: > I was going to commit the first of these patches, but then I noticed Andres > said he was planning to, so I'll wait another day. I'd be happy for you to take this on...

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

On Tue, Aug 9, 2022 at 9:20 AM Tom Lane wrote: > > Thomas Munro writes: > > On Tue, Aug 9, 2022 at 3:27 PM Tom Lane wrote: > >> I have not tried to analyze the error-handling properties of 0001, > >> but if it's being equally cavalier then it shouldn't be committed > >> either. > > > 0001 does

Re: optimize lookups in snapshot [sub]xip arrays

On Tue, Aug 09, 2022 at 10:57:44AM +0900, Masahiko Sawada wrote: > The patch looks good to me. One minor point is: Thanks for taking a look. > + * IDENTIFICATION > + * src/port/pg_lfind.h > > The path doesn't match to the actual file path, src/include/port/pg_lfind.h. Fixed in v10. --

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

Thomas Munro writes: > On Tue, Aug 9, 2022 at 3:27 PM Tom Lane wrote: >> I have not tried to analyze the error-handling properties of 0001, >> but if it's being equally cavalier then it shouldn't be committed >> either. > 0001 does introduce new errors, as mentioned in the commit message, in >

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

On Tue, Aug 9, 2022 at 3:27 PM Tom Lane wrote: > Actually, having now read the patch, I don't think there is any > part of 0002 that is a good idea. It's blithely removing the > comments that explain why the existing coding is the way it is, > and not providing a shred of justification for

Re: out of date comment in commit_ts.c

On Tue, Aug 09, 2022 at 01:02:15PM +1200, Thomas Munro wrote: > Pushed. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

I wrote: > Thomas Munro writes: >> The only hunk I'm having second thoughts about is the following, which >> makes unexpected stray files break checkpoints: > Sounds like a pretty bad idea. What's the upside? Actually, having now read the patch, I don't think there is any part of 0002 that is

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

Thomas Munro writes: > The only hunk I'm having second thoughts about is the following, which > makes unexpected stray files break checkpoints: Sounds like a pretty bad idea. What's the upside? regards, tom lane

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

Working on get_dirent_type() reminded me of this thread. I was going to commit the first of these patches, but then I noticed Andres said he was planning to, so I'll wait another day. Here they are, with commit messages but otherwise unchanged from Nathan's v12 except for a slight comment tweak:

Re: pg15b2: large objects lost on upgrade

Bruce Momjian writes: > I thought the test was setting up a configuration that would never be > used by normal servers. Is that false? *If* we made it disable autovac before starting pg_upgrade, then that would be a process not used by normal users. I don't care whether pg_upgrade disables

Re: Fix unmatched file identifications

On Tue, Aug 9, 2022 at 11:24 AM John Naylor wrote: > > On Tue, Aug 9, 2022 at 8:58 AM Masahiko Sawada wrote: > > I found that there are two .c and .h files whose identification in the > > header comment doesn't match its actual path. > > > The attached small patch fixes them. > > Pushed, thanks!

Re: pg15b2: large objects lost on upgrade

On Mon, Aug 8, 2022 at 09:51:46PM -0400, Tom Lane wrote: > Bruce Momjian writes: > >> Hmmm ... now that you mention it, I see nothing in 002_pg_upgrade.pl > >> that attempts to turn off autovacuum on either the source server or > >> the destination. So one plausible theory is that autovac moved

Re: Fix unmatched file identifications

On Tue, Aug 9, 2022 at 8:58 AM Masahiko Sawada wrote: > I found that there are two .c and .h files whose identification in the > header comment doesn't match its actual path. > The attached small patch fixes them. Pushed, thanks! -- John Naylor EDB: http://www.enterprisedb.com

Re: optimize lookups in snapshot [sub]xip arrays

On Tue, Aug 9, 2022 at 7:33 AM Nathan Bossart wrote: > > On Mon, Aug 08, 2022 at 01:46:48PM +0700, John Naylor wrote: > > Okay, I think it's basically in good shape. Since it should be a bit > > faster than a couple versions ago, would you be up for retesting with > > the original test having 8

Fix unmatched file identifications

Hi, I found that there are two .c and .h files whose identification in the header comment doesn't match its actual path. src/include/common/compression.h has: * IDENTIFICATION *src/common/compression.h *-

Re: pg15b2: large objects lost on upgrade

Bruce Momjian writes: >> Hmmm ... now that you mention it, I see nothing in 002_pg_upgrade.pl >> that attempts to turn off autovacuum on either the source server or >> the destination. So one plausible theory is that autovac moved the >> numbers since we checked. > Uh, pg_upgrade assumes

Re: out of date comment in commit_ts.c

On Thu, Jul 28, 2022 at 8:30 AM Nathan Bossart wrote: > On Tue, Jul 26, 2022 at 10:33:43AM -0700, Nathan Bossart wrote: > > IIUC the ability for callers to request WAL record generation is no longer > > possible as of 08aa89b [0]. Should the second sentence be removed? > > Here's a patch.

Re: pg15b2: large objects lost on upgrade

On Tue, Aug 2, 2022 at 03:32:05PM -0400, Tom Lane wrote: > "Jonathan S. Katz" writes: > > On 8/2/22 1:12 PM, Tom Lane wrote: > >> Sadly, we're still not out of the woods. I see three buildfarm > >> failures in this test since Robert resolved the "-X" problem [1][2][3]: > > > Looking at the

Re: Cleaning up historical portability baggage

Thomas Munro writes: > Yeah, that's not true anymore, and QueryPerformanceCounter() is faster > than Get­System­Time­Precise­As­File­Time()[1], but there doesn't > really seem to be any point in mentioning that or gettimeofday() at > all here. I propose to cut it down to just: > * This file

Re: Cleaning up historical portability baggage

On Mon, Aug 8, 2022 at 12:27 PM Tom Lane wrote: > BTW, that commit really should have updated the explanation at the top of > instr_time.h: > > * This file provides an abstraction layer to hide portability issues in > * interval timing. On Unix we use clock_gettime() if available, else > *

Re: Temporary file access API

On Mon, Aug 8, 2022 at 08:27:27PM +0200, Antonin Houska wrote: > > For instance, to go back to my earlier comments, if the data is > > already block-structured, we do not need to insert a second layer of > > buffering; if it isn't, maybe we should, not just for TDE but for > > other reasons as

Re: conchuela doesn't like gnu_printf anymore

=?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: > Yes, when I upgraded to the lastest DragonFly BSD 6.2.2 I was meaning to > switch to CLANG14 for both C and C++. I guess I fat fingered the > configuration somehow. > I have switch to CLANG14 for both C and C++ now. > Let's see if the warnings

Re: optimize lookups in snapshot [sub]xip arrays

On Mon, Aug 08, 2022 at 12:56:28PM +0530, Bharath Rupireddy wrote: > 1) pg_lfind32 - why just uint32? If it's not possible to define > functions for char, unsigned char, int16, uint16, int32, int64, uint64 > and so on, can we add a few comments around that? Also, the comments > can talk about if

Re: Checking pgwin32_is_junction() errors

On Tue, Aug 9, 2022 at 8:30 AM Thomas Munro wrote: > On Mon, Aug 8, 2022 at 8:23 PM wrote: > > "C:/HOME" is the junction point to the second volume on my hard drive - > > "\??\Volume{GUID}\" which name pgreadlink() erroneously strips here: > >

Re: optimize lookups in snapshot [sub]xip arrays

On Mon, Aug 08, 2022 at 01:46:48PM +0700, John Naylor wrote: > Okay, I think it's basically in good shape. Since it should be a bit > faster than a couple versions ago, would you be up for retesting with > the original test having 8 to 512 writers? Sure thing. The results are similar. As

Re: logical replication restrictions

On Wed, Aug 3, 2022, at 10:27 AM, Amit Kapila wrote: > Your explanation makes sense to me. The other point to consider is > that there can be cases where we may not apply operation for the > transaction because of empty transactions (we don't yet skip empty > xacts for prepared transactions). So,

dropping datumSort field

Hi, I was looking at ExecSort() w.r.t. datum sort. I wonder if the datumSort field can be dropped. Here is a patch illustrating the potential simplification. Please take a look. Thanks drop-datum-sort-field.patch Description: Binary data

Re: logical replication restrictions

On Wed, Jul 13, 2022, at 2:34 PM, Melih Mutlu wrote: [Sorry for the delay...] > 22. src/test/subscription/t/032_apply_delay.pl >> >> I received the following error when trying to run these 'subscription' tests: >> >> t/032_apply_delay.pl ... No such class log_location at >>

Re: Checking pgwin32_is_junction() errors

On Mon, Aug 8, 2022 at 8:23 PM wrote: > initdb on my windows 10 system stopped working after the commit > c5cb8f3b: "Provide lstat() for Windows." > The error message is: creating directory C:/HOME/data ... initdb: > error: could not create directory "C:/HOME": File exists > > "C:/HOME" is the

Re: Support logical replication of DDLs

> In general, I agree with your comment below that we can work on this > after we have some more concrete plans/discussions. I think we can > probably consider this when we have more discussion around the > publication commands for the DDL objects. However, it would be good if > you can add some

Support logical replication of global object commands

Hello, Logical replication of DDL commands support is being worked on in [1]. However, global object commands are quite different from other non-global object DDL commands and need to be handled differently. For example, global object commands include ROLE statements, DATABASE statements,

Re: conchuela doesn't like gnu_printf anymore

On 2022-08-06 18:59, Tom Lane wrote: This is seemingly an intentional configuration change, because the animal is reporting different config_env than before. However, we decide what to set PG_PRINTF_ATTRIBUTE to based on what CC likes, and if CXX doesn't like it then you'll get these

Re: Patch to address creation of PgStat* contexts with null parent context

Hi, On 2022-08-08 15:12:08 +0900, Kyotaro Horiguchi wrote: > At Sat, 6 Aug 2022 19:19:39 -0700, Andres Freund wrote in > > Hi, > > > > On 2022-08-05 17:22:38 +0900, Kyotaro Horiguchi wrote: > > > I think it a bit different. Previously that memory (but for a bit > > > different use, precisely)

Managing my own index partitions

(resending because I sent the original from the wrong email address...) I'm building a Postgres index access method. For a variety of reasons it's more efficient to store the index data in multiple physical files on disk rather than in the index's main fork. I'm trying to create separate rels

Re: automatically generating node support functions

I wrote: > Ah. It'd help if that complaint said what the command input actually > is :-(. But on looking closer, I missed stripping the empty strings > that "split" will produce at the ends of the array. I think the > attached will do the trick, and I really do want to get rid of this > copy of

Managing my own index partitions

I'm building a Postgres index access method. For a variety of reasons it's more efficient to store the index data in multiple physical files on disk rather in the index's main fork. I'm trying to create separate rels that can be created and destroyed by the parent index access method. I've

Re: 2022-08-11 release announcement draft

On 8/8/22 12:44 PM, Justin Pryzby wrote: On Mon, Aug 08, 2022 at 11:50:16AM -0400, Jonathan S. Katz wrote: * Fix [`pg_upgrade`](https://www.postgresql.org/docs/current/pgupgrade.html) to detect non-upgradable usages of functions accepting `anyarray` parameters. use or usage This line comes

Re: bug on log generation ?

> > OK, we really need a repeatable test if possible. Perhaps a pgbench run > with lots of concurrent runs of a some very long query would do the trick. > OK, I can do it but ... strangely that error usually occurs at random times, sometimes at 08:00, sometimes at 19:00, and it's busier between

Re: Temporary file access API

Robert Haas wrote: > On Fri, Jul 29, 2022 at 11:36 AM Antonin Houska wrote: > > Attached is a new version. It allows the user to set "elevel" (i.e. ERROR is > > not necessarily thrown on I/O failure, if the user prefers to check the > > number > > of bytes read/written) and to specify the

Re: automatically generating node support functions

Amit Kapila writes: > On Sun, Aug 7, 2022 at 8:19 PM Tom Lane wrote: >> Meh ... it's not checking the data files themselves. Here's >> a patch based on the logic for invoking genbki. Completely >> untested, would somebody try it? > I tried it on commit a69959fab2 just before the commit

Patch to provide example for ssl certification authentication

I couldn't find a clear document which showed how this was done. The example would help. Dave Cramer ssl_cert_auth.patch Description: Binary data

Re: make update-po@master stops at pg_upgrade

On 2022-Aug-08, Tom Lane wrote: > Alvaro Herrera writes: > > Now that I did the translation chores again after a few years I am > > reminded of a point about this argument: in reality, few people ever > > run this recipe manually (I know I never do), because it's easier to > > grab the

Re: Get the statistics based on the application name and IP address

Hi, On Mon, Aug 08, 2022 at 08:21:06PM +0500, Ibrar Ahmed wrote: > While working on pg_stat_stements, I got some questions from customers to > have statistics by application and IP address. > [...] > name. I did some POC and had a patch. But before sharing the patch. > > I need to know if there

Re: Asking for feedback on Pgperffarm

Hi Yedil, On Mon, Aug 08, 2022 at 02:50:17PM +0200, Yedil Serzhan wrote: > Dear hackers, > > I'm Yedil. I'm working on the project "Postgres Performance Farm" during > Gsoc. Pgperffarm is a project like Postgres build farm but focuses on the > performance of the database. Now it has 2 types of

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

On Mon, Aug 8, 2022 at 9:17 AM Matthias van de Meent wrote: > Because if a subset of the pages of a relation contains more tuples > than your current total expected tuples in the table, you should > update your expectations regardless of which blocks or which number of > blocks you've scanned -

Re: 2022-08-11 release announcement draft

On Mon, Aug 08, 2022 at 11:50:16AM -0400, Jonathan S. Katz wrote: > * Fix [`pg_upgrade`](https://www.postgresql.org/docs/current/pgupgrade.html) > to > detect non-upgradable usages of functions accepting `anyarray` parameters. use or usage > and regressions before the general availability of

Re: bug on log generation ?

Hi, On 2022-08-08 12:19:22 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2022-08-08 10:32:22 -0400, Tom Lane wrote: > >> Another idea is that some of the write() calls are failing --- elog.c > >> doesn't check for that. > > > I was suspicious of those as well. It might be a good idea to

Re: make update-po@master stops at pg_upgrade

Alvaro Herrera writes: > On 2022-Jul-28, Tom Lane wrote: >> Thanks for testing it. I think the only remaining concern is >> Peter's objection that $(wildcard) might pick up random junk files >> that end in ".c". That's true, but the backend's nls.mk also >> picks up everything matching "*.c"

Re: bug on log generation ?

Andres Freund writes: > On 2022-08-08 10:32:22 -0400, Tom Lane wrote: >> Another idea is that some of the write() calls are failing --- elog.c >> doesn't check for that. > I was suspicious of those as well. It might be a good idea to at least write > such failures to stderr, otherwise it's just

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

On Mon, 8 Aug 2022 at 17:49, Peter Geoghegan wrote: > > On Mon, Aug 8, 2022 at 8:33 AM Matthias van de Meent > wrote: > > For example, if currently the measured 2% of the pages contains more > > than 100% of the previous count of tuples, or with your patch the last > > page contains more than

Re: make update-po@master stops at pg_upgrade

On 2022-Jul-28, Tom Lane wrote: > Thanks for testing it. I think the only remaining concern is > Peter's objection that $(wildcard) might pick up random junk files > that end in ".c". That's true, but the backend's nls.mk also > picks up everything matching "*.c" (over the whole backend tree, >

Re: bug on log generation ?

Hi, On 2022-08-08 10:32:22 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > What platform is this on? Is it possible that on some platform the chunk > > size we're using is not doing an atomic write? > > Another idea is that some of the write() calls are failing --- elog.c > doesn't check

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

> The server seem to have started as a standby after crashing a > primary. Is it correct? Yes, that is correct. 2022-08-05 17:18:51 UTC::@:[359]:LOG: database system was interrupted; last known up at 2022-08-05 17:08:52 UTC 2022-08-05 17:18:51 UTC::@:[359]:LOG: creating missing WAL directory

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

Hi, Thanks for the feedback, see my reply below. > > > It turns out that we already invalidate the relevant entries in > LogicalRepRelMap/LogicalRepPartMap when "ANALYZE" (or VACUUM) updates any > of the statistics in pg_class. > > > > The call-stack for analyze is roughly: > > do_analyze_rel()

Re: failing to build preproc.c on solaris with sun studio

Hi, On 2022-08-08 11:14:58 -0400, Andrew Dunstan wrote: > I'm waiting for that first commit so I can start working on the > buildfarm client changes. Ideally (from my POV) this would happen by > early Sept when I will be leaving on a trip for some weeks, and this > would be a good project to take

Re: shared-memory based stats collector - v70

I'm trying to wrap my head around the shared memory stats collector infrastructure from <20220406030008.2qxipjxo776dw...@alap3.anarazel.de> committed in 5891c7a8ed8f2d3d577e7eea34dacff12d7b6bbd. I have one question about locking -- afaics there's nothing protecting reading the shared memory

Re: [RFC] building postgres with meson - v10

Hi, On 2022-07-21 15:26:05 +0300, Bilal Yavuz wrote: > > On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote: > > > dff7b5a960 meson: prereq: regress: allow to specify director containing > > > expected files. > > > > > > This could use a bit more explanation, but it doesn't look > > >

2022-08-11 release announcement draft

Hi, Please see attached draft of the 2022-08-11 release announcement. Please provide feedback on {technical accuracy, omissions, any other errors} no later than 2022-08-11 0:00 AoE[1]. Thanks, Jonathan [1] https://en.wikipedia.org/wiki/Anywhere_on_Earth The PostgreSQL Global Development

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

On Mon, Aug 8, 2022 at 8:33 AM Matthias van de Meent wrote: > For example, if currently the measured 2% of the pages contains more > than 100% of the previous count of tuples, or with your patch the last > page contains more than 100% of the previous count of the tuples, that > new count is

Re: [PATCH] CF app: add "Returned: Needs more interest"

Hi, On 2022-08-08 08:37:41 -0700, Jacob Champion wrote: > Agreed. This probably bleeds over into the other documentation thread > a bit -- how do we want to communicate the subtle points to people in > a CF? We should write a docs patch for it and then reference if from a bunch of places. I

Re: [PATCH] CF app: add "Returned: Needs more interest"

On Thu, Aug 4, 2022 at 3:00 PM Andres Freund wrote: > On 2022-08-04 11:19:28 -0700, Jacob Champion wrote: > > My intention had not quite been for this to be a referendum on the > > decision for every patch -- we can do that if it helps, but I don't > > think we necessarily have to have unanimity

Re: bug on log generation ?

On 2022-08-08 Mo 11:07, Marcos Pegoraro wrote: > > What platform is this on? Is it possible that on some platform the > chunk > size we're using is not doing an atomic write? > >   > Until last year we were Ubuntu 16.04 and Postgres 11 with the latest > minor update. > This January

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

On Mon, 8 Aug 2022 at 17:26, Peter Geoghegan wrote: > > On Mon, Aug 8, 2022 at 8:14 AM Matthias van de Meent > wrote: > > I do not have intimate knowledge of this code, but shouldn't we also > > add some sefety guarantees like the following in these blocks? Right > > now, we'll keep

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

On Mon, Aug 8, 2022 at 8:14 AM Matthias van de Meent wrote: > I do not have intimate knowledge of this code, but shouldn't we also > add some sefety guarantees like the following in these blocks? Right > now, we'll keep underestimating the table size even when we know that > the count is

Get the statistics based on the application name and IP address

While working on pg_stat_stements, I got some questions from customers to have statistics by application and IP address. I know that we are collecting the statistics by query id, user id, database id and top-level query. There is no way to collect the statistics based on IP address and application

Re: Clarifying Commitfest policies

On Thu, 4 Aug 2022 at 20:38, Jacob Champion wrote: > > On Wed, Aug 3, 2022 at 2:05 PM Matthias van de Meent > wrote: > > On Wed, 3 Aug 2022 at 20:04, Jacob Champion wrote: > > > Is that enough, or should we do more? > > > > "The CF Checklist" seems to refer to only the page that is (or seems >

Re: failing to build preproc.c on solaris with sun studio

On 2022-08-07 Su 02:46, Andres Freund wrote: > >> I think if we want to get this past the finish line, we need to >> acknowledge that the initial commit isn't going to be perfect. >> The whole point of continuing to maintain the Makefiles is to >> give us breathing room to fix remaining issues

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

On Mon, 8 Aug 2022 at 16:52, Peter Geoghegan wrote: > > On Fri, Aug 5, 2022 at 5:39 PM Peter Geoghegan wrote: > > Attached patch fixes closes the remaining gap. With the patch applied, > > the second and subsequent vacuum verbose operations from the test case > > will show that reltuples is

Re: bug on log generation ?

> > What platform is this on? Is it possible that on some platform the chunk > size we're using is not doing an atomic write? > Until last year we were Ubuntu 16.04 and Postgres 11 with the latest minor update. This January we changed to Ubuntu 20.04 and Postgres 14, now updated to 14.4. But the

Re: `make check` doesn't pass on MacOS Catalina

On 2022-08-06 Sa 12:10, Andrew Dunstan wrote: > On 2022-08-06 Sa 11:25, Tom Lane wrote: >> Andrew Dunstan writes: >>> I came across this when I was working on setting up some Dockerfiles for >>> the buildfarm. Apparently LD_LIBRARY_PATH doesn't work on Alpine, at >>> least out of the box, as it

Re: Remaining case where reltuples can become distorted across multiple VACUUM operations

On Fri, Aug 5, 2022 at 5:39 PM Peter Geoghegan wrote: > Attached patch fixes closes the remaining gap. With the patch applied, > the second and subsequent vacuum verbose operations from the test case > will show that reltuples is still 1 (it won't ever change). The > patch just extends an old

Re: bug on log generation ?

Andrew Dunstan writes: > What platform is this on? Is it possible that on some platform the chunk > size we're using is not doing an atomic write? Another idea is that some of the write() calls are failing --- elog.c doesn't check for that. Eyeing the POSIX spec for write(), I wonder if somehow

Re: bug on log generation ?

On 2022-08-08 Mo 07:34, Marcos Pegoraro wrote: > > > How are you running postgres? If the logger process runs into > trouble it might > write to stderr. > > Is there a chance your huge statements would make you run out of > space? > > Well, I don't think it is a  out of space

Refactor backup related code (was: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?)

On Sat, Jul 30, 2022 at 5:37 AM Bharath Rupireddy wrote: > > On Tue, Jul 26, 2022 at 5:50 PM David Steele wrote: > > > > >> I would prefer to have all the components of backup_label stored > > >> separately and then generate backup_label from them in pg_backup_stop(). > > > > > > +1, because

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

On Fri, Aug 5, 2022 at 8:19 AM Kyotaro Horiguchi wrote: > > At Tue, 26 Apr 2022 08:26:59 +0200, Laurenz Albe > wrote in > > While this may mitigate the problem, I don't think it will deal with > > all the cases which could cause a transaction to end up committed locally, > > but not on the

SUBTRANS: Minimizing calls to SubTransSetParent()

On Thu, 4 Aug 2022 at 13:11, Simon Riggs wrote: > > On Wed, 3 Aug 2022 at 20:18, Andres Freund wrote: > > > I think we should consider redesigning subtrans more substantially - even > > with > > the changes you propose here, there's still plenty ways to hit really bad > > performance. And

Asking for feedback on Pgperffarm

Dear hackers, I'm Yedil. I'm working on the project "Postgres Performance Farm" during Gsoc. Pgperffarm is a project like Postgres build farm but focuses on the performance of the database. Now it has 2 types of benchmarks, pgbench and tpc-h. The website is online here ,

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

On Sun, Aug 7, 2022 at 9:22 PM Bharath Rupireddy wrote: > > On Sun, Aug 7, 2022 at 3:19 PM Thomas Munro wrote: > > > > > A second thing is that pg_pwritev_with_retry_and_write_zeros() is > > > designed to work on WAL segments initialization and it uses > > > XLOG_BLCKSZ and PGAlignedXLogBlock

Re: Reducing planning time on tables with many indexes

On 27.07.22, 18:39, "Tom Lane" wrote: [External Email] David Geier writes: > We tracked down the root cause of this slowdown to lock contention in > 'get_relation_info()'. The index lock of every single index of every single > table used in that query is acquired. We

Re: bug on log generation ?

> > > How are you running postgres? If the logger process runs into trouble it > might > write to stderr. > > Is there a chance your huge statements would make you run out of space? > > Well, I don't think it is a out of space problem, because it doesn´t stop logging, it just splits that message.

Re: Skipping schema changes in publication

On Mon, Aug 8, 2022 at 12:46 PM vignesh C wrote: > > On Fri, Jun 3, 2022 at 3:36 PM vignesh C wrote: > > > > On Thu, May 26, 2022 at 7:04 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > On Monday, May 23, 2022 2:13 PM vignesh C wrote: > > > > Attached v7 patch which fixes the buildfarm

Re: optimize lookups in snapshot [sub]xip arrays

On Mon, Aug 8, 2022 at 2:30 PM John Naylor wrote: > > On Mon, Aug 8, 2022 at 2:26 PM Bharath Rupireddy > wrote: > > > 3) Can pg_lfind32 return the index of the key found, for instance to > > use it for setting/resetting the found element in the array? > > That was just discussed. It's slightly

Re: optimize lookups in snapshot [sub]xip arrays

On Mon, Aug 8, 2022 at 2:26 PM Bharath Rupireddy wrote: > > > 1) pg_lfind32 - why just uint32? If it's not possible to define > functions for char, unsigned char, int16, uint16, int32, int64, uint64 > and so on, can we add a few comments around that? Also, the comments Future work, as far as I'm

Re: logical decoding and replication of sequences

At Mon, 08 Aug 2022 17:33:22 +0900 (JST), Kyotaro Horiguchi wrote in > If WaitForWALToBecomeAvailable returned by promotion, ReadPageInteral > misses the chance to inavlidate reader-state. That state is not an > error while in StandbyMode. Mmm... Maybe I wanted to say: (Still I'm not sure

Re: Column Filtering in Logical Replication

PSA patch version v1* for a new "Column Lists" pgdocs section This is just a first draft, but I wanted to post it as-is, with the hope that I can get some feedback while continuing to work on it. -- Kind Regards, Peter Smith. Fujitsu Australia

Re: logical decoding and replication of sequences

At Mon, 8 Aug 2022 18:15:46 +1200, Thomas Munro wrote in > On Mon, Aug 8, 2022 at 9:09 AM Thomas Munro wrote: > > Thanks for the repro patch and bisection work. Looking... > > I don't have the complete explanation yet, but it's something like > this. We hit the following branch in

Re: Checking pgwin32_is_junction() errors

On 2022-08-06 08:02, Thomas Munro wrote: Pushed. Hmm, this stuff could *really* use a little test framework that's run by check-world, that exercises these various replacement operations. But I also suspect that problems in this area are likely to be due to concurrency. It's hard to make a

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

Hi Andres, On Sat, Aug 6, 2022 at 5:37 AM Andres Freund wrote: > On 2022-08-04 17:01:48 +0900, Amit Langote wrote: > > On Wed, Aug 3, 2022 at 12:00 AM Andres Freund wrote: > > > Honestly, this code seems like it should just be rewritten from scratch. > > > > Based on what I wrote above, please

Re: optimize lookups in snapshot [sub]xip arrays

On Mon, Aug 8, 2022 at 12:17 PM John Naylor wrote: > > On Sun, Aug 7, 2022 at 4:25 AM Nathan Bossart > wrote: > > > > [v8] > > Okay, I think it's basically in good shape. Since it should be a bit > faster than a couple versions ago, would you be up for retesting with > the original test having

Re: Skipping schema changes in publication

On Fri, Jun 3, 2022 at 3:36 PM vignesh C wrote: > > On Thu, May 26, 2022 at 7:04 PM osumi.takami...@fujitsu.com > wrote: > > > > On Monday, May 23, 2022 2:13 PM vignesh C wrote: > > > Attached v7 patch which fixes the buildfarm warning for an unused warning > > > in > > > release mode as in

Re: automatically generating node support functions

On Sun, Aug 7, 2022 at 8:19 PM Tom Lane wrote: > > Amit Kapila writes: > > On Wed, Aug 3, 2022 at 7:16 PM Tom Lane wrote: > >> More likely, we need to add something explicit to Mkvcbuild.pm > >> for this. I recall that it has stanzas to deal with updating > >> other autogenerated files; I bet

Re: optimize lookups in snapshot [sub]xip arrays

On Sun, Aug 7, 2022 at 4:25 AM Nathan Bossart wrote: > > [v8] Okay, I think it's basically in good shape. Since it should be a bit faster than a couple versions ago, would you be up for retesting with the original test having 8 to 512 writers? And also add the const markers we discussed

Re: Logical replication - schema change not invalidating the relation cache

On Sat, Mar 12, 2022 at 1:29 PM vignesh C wrote: > > On Fri, Dec 3, 2021 at 3:21 PM vignesh C wrote: > > > > On Fri, Dec 3, 2021 at 1:13 PM Michael Paquier wrote: > > > > > > On Thu, Aug 26, 2021 at 09:00:39PM +0530, vignesh C wrote: > > > > The previous patch was failing because of the recent

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

On Thu, Aug 4, 2022 at 11:59 AM Bharath Rupireddy wrote: > > On Sun, Jul 31, 2022 at 8:36 PM mahendrakar s > wrote: > > > >> On Mon, 25 Jul 2022 at 16:42, Bharath Rupireddy > >> wrote: > >> Here's the v3 patch after rebasing. > >> > >> I just would like to reiterate the issue the patch is

  1   2   >