Re: [PATCH] Runtime control of CLOBBER_CACHE_ALWAYS

2020-12-02 Thread Craig Ringer
On Tue, 27 Oct 2020 at 16:34, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-09-25 09:40, Craig Ringer wrote: > > While working on extensions I've often wanted to enable cache clobbering > > for a targeted piece of code, without paying the price of r

Re: Two fsync related performance issues?

2020-12-01 Thread Craig Ringer
On Wed, 14 Oct 2020, 13:06 Michael Paquier, wrote: > On Wed, Oct 14, 2020 at 02:48:18PM +1300, Thomas Munro wrote: > > On Wed, Oct 14, 2020 at 12:53 AM Michael Banck > > wrote: > >> One question about this: Did you consider the case of a basebackup being > >> copied/restored somewhere and the

Re: [PATCH] Runtime control of CLOBBER_CACHE_ALWAYS

2020-12-01 Thread Craig Ringer
On Mon, 30 Nov 2020, 20:38 Anastasia Lubennikova, < a.lubennik...@postgrespro.ru> wrote: > On 27.10.2020 11:34, Peter Eisentraut wrote: > > On 2020-09-25 09:40, Craig Ringer wrote: > >> While working on extensions I've often wanted to enable cache > >> clobbe

Re: TAP test utility module 'PG_LSN.pm'

2020-11-30 Thread Craig Ringer
On Tue, 1 Dec 2020 at 13:58, Michael Paquier wrote: > > On Tue, Dec 01, 2020 at 12:03:41PM +0800, Craig Ringer wrote: > > I'd like to share the attached PG_LSN.pm module that I use when > > writing TAP tests. I suggest that it be considered for inclusion in > > core. &

TAP test utility module 'PG_LSN.pm'

2020-11-30 Thread Craig Ringer
Hi all I'd like to share the attached PG_LSN.pm module that I use when writing TAP tests. I suggest that it be considered for inclusion in core. It defines a Perl datatype PG_LSN with operator support, so you can write things like cmp_ok($got_lsn, "<", $expected_lsn, "testname") in TAP

Re: Printing backtrace of postgres processes

2020-11-30 Thread Craig Ringer
On Tue, 1 Dec 2020 at 11:31, Andres Freund wrote: > > Hi, > > On 2020-11-30 13:35:46 +0800, Craig Ringer wrote: > > I find that when I most often want a backtrace of a running, live > > backend, it's because the backend is doing something that isn't > > passing a

Re: Printing backtrace of postgres processes

2020-11-30 Thread Craig Ringer
On Tue, 1 Dec 2020 at 07:04, Tom Lane wrote: > I'd feel better about it if the mechanism had you specify exactly > one target process, and were restricted to a superuser requestor. Er, rather. I actually assumed the former was the case already, not having looked closely yet.

Re: proposal: function pg_setting_value_split() to parse shared_preload_libraries etc.

2020-11-29 Thread Craig Ringer
On Mon, 30 Nov 2020 at 03:24, Anastasia Lubennikova wrote: > > On 23.10.2020 05:06, Ian Lawrence Barwick wrote: > > Having just submitted this, I realised I'm focussing on the GUCs which call > > "SplitDirectoriesString()" (as my specific uses case is for > > "shared_preload_libraries") > > but

Re: Printing backtrace of postgres processes

2020-11-29 Thread Craig Ringer
> Surely this is *utterly* unsafe. You can't do that sort of stuff in > a signal handler. Not safely, anyway. The signal handler could be called in the middle of a malloc(), a pfree(), or all sorts of other exciting circumstances. It'd have to be extremely careful to use only local resources on

Notes on physical replica failover with logical publisher or subscriber

2020-11-29 Thread Craig Ringer
Hi all I recently wrote some notes on interaction between physical replication failover/promotion and logical replication publisher and/or standby. As you probably all know, right now we don't support physical failover for logical replication publishers at all, either for in-core logical

RFC: extension_search_path to supplement dynamic_library_path

2020-11-29 Thread Craig Ringer
Hi all We have a dynamic_library_path setting to allow postgres to search additional locations for loadable libraries (since 8.1, commit 761a0bb69b). This permits libraries to be located in alternate locations for loading by shared_preload_libraries, LOAD commands, implicit loading via function

Re: Asynchronous Append on postgres_fdw nodes.

2020-11-29 Thread Craig Ringer
"On Thu, Nov 26, 2020 at 9:28 AM movead...@highgo.ca wrote: > > > I test the patch and occur several issues as blow: > > Issue one: > Get a Assert error at 'Assert(bms_is_member(i, node->as_needrequest));' in > ExecAppendAsyncRequest() function when I use more than two foreign table > on

Re: POC: postgres_fdw insert batching

2020-11-29 Thread Craig Ringer
On Fri, 27 Nov 2020, 14:06 tsunakawa.ta...@fujitsu.com, wrote: > > Also, I'm afraid it requires major surgery or reform of executor. I don't want it to delay the release of reasonably good (10x) improvement with the synchronous interface.) Totally sensible. If it isn't feasible without

Re: Printing LSN made easy

2020-11-29 Thread Craig Ringer
On Mon, Nov 30, 2020 at 1:10 AM Tom Lane wrote: > Michael Paquier writes: > > On Fri, Nov 27, 2020 at 04:10:27PM +0530, Ashutosh Bapat wrote: > >> Off list Craig Ringer suggested introducing a new format specifier > >> similar to %m for LSN but I did

Re: Add docs stub for recovery.conf

2020-11-29 Thread Craig Ringer
On Sat, Nov 14, 2020 at 1:42 AM Bruce Momjian wrote: > > Clearly we have need for documenting these renamings somewhere. We were > going to go with a simple URL redirect and a "tip" for > default/pre-installed roles, but I like the idea of doing something more > wholistic that covers all of our

Re: POC: postgres_fdw insert batching

2020-11-27 Thread Craig Ringer
On Sat, 28 Nov 2020, 10:10 Tomas Vondra, wrote: > > > On 11/27/20 7:05 AM, tsunakawa.ta...@fujitsu.com wrote: > > However, the FDW interface as it's implemented today is not designed to > allow that, I believe (we pretty much just invoke the FWD callbacks as > if it was a local AM). It assumes

Re: POC: postgres_fdw insert batching

2020-11-26 Thread Craig Ringer
On Fri, Nov 27, 2020 at 10:47 AM tsunakawa.ta...@fujitsu.com < tsunakawa.ta...@fujitsu.com> wrote: Covering this one first: I expect postgresExecForeignBatchInsert() would be able to use the libpq > batching API, because it receives an array of tuples and can generate and > issue INSERT

Re: POC: postgres_fdw insert batching

2020-11-26 Thread Craig Ringer
On Fri, Nov 27, 2020 at 3:34 AM Tomas Vondra wrote: > Not sure how is this related to app developers? I think the idea was > that the libpq features might be useful between the two PostgreSQL > instances. I.e. the postgres_fdw would use the libpq batching to send > chunks of data to the other

Re: [PATCH] LWLock self-deadlock detection

2020-11-25 Thread Craig Ringer
On Wed, Nov 25, 2020 at 9:23 PM Ashutosh Bapat wrote: > On Wed, Nov 25, 2020 at 11:47 AM Craig Ringer > wrote: > >> I am also seeing a pattern > >> Assert(!LWLockHeldByMe()); > >> LWLockAcquire() > >> > >> at some places. Should we cha

Re: [PATCH] LWLock self-deadlock detection

2020-11-24 Thread Craig Ringer
On Tue, Nov 24, 2020 at 10:11 PM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > This looks useful. LWLockCheckSelfDeadlock() could use LWLockHeldByMe > variant instead of copying that code with possibly a change in that > function to return the required information. > Yes, possibly so.

Re: POC: postgres_fdw insert batching

2020-11-24 Thread Craig Ringer
On Thu, Oct 8, 2020 at 10:40 AM tsunakawa.ta...@fujitsu.com < tsunakawa.ta...@fujitsu.com> wrote: > > Thank you for picking up this. I'm interested in this topic, too. (As an > aside, we'd like to submit a bulk insert patch for ECPG in the near future.) > > As others referred, Andrey-san's fast

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-24 Thread Craig Ringer
On Wed, Nov 25, 2020 at 2:43 AM Alexey Kondratov wrote: > On 2020-11-24 06:52, Bharath Rupireddy wrote: > > Thanks for the review comments. > > > > On Mon, Nov 23, 2020 at 9:57 PM Alexey Kondratov > > wrote: > >> > >> > v1-0001-postgres_fdw-function-to-discard-cached-connections.patch > >> > >>

Re: ResourceOwner refactoring

2020-11-19 Thread Craig Ringer
very in an extension mainloop without copy/pasting a bunch of PostgresMain code, better default signal handlers, startup/shutdown that shares more with user backends, etc. Right now it's quite tricky to get bgworkers to behave well. From 9f3e6978b1bbfeb8ca3f1cac42e86c4731143404 Mon Sep 17 00:00:0

Re: Add LWLock blocker(s) information

2020-11-19 Thread Craig Ringer
ntly fire T_NAME(lock) which calls GetLWTrancheName() for each tracepoint hit, so long as Pg is built with --enable-dtrace, even when nothing is actually tracing them. We might want to consider guarding them in systemtap tracepoint semaphore tests so they just become a predicted-away branch when not

[PATCH] LWLock self-deadlock detection

2020-11-19 Thread Craig Ringer
if nobody likes the idea of merging it. As written the check runs only for --enable-cassert builds or when LOCK_DEBUG is defined. From 0ec0beb294f4c5ed35dbb35260f53b069563638f Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Thu, 19 Nov 2020 14:24:55 +0800 Subject: [PATCH] LWLock self-deadlock detection

Re: Detecting File Damage & Inconsistencies

2020-11-17 Thread Craig Ringer
On Fri, Nov 13, 2020 at 7:24 PM Simon Riggs wrote: > > What I'm proposing is an option to add 16 bytes onto each COMMIT > record > Would it make sense to write this at the time we write a topxid assignment to WAL instead? Otherwise it won't be accessible to streaming-mode logical decoding.

Re: abstract Unix-domain sockets

2020-11-17 Thread Craig Ringer
On Fri, Oct 9, 2020 at 3:28 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > During the discussion on Unix-domain sockets on Windows, someone pointed > out[0] abstract Unix-domain sockets. > This reminds me on a somewhat random note that SSPI mode authentication should work out

Re: Add docs stub for recovery.conf

2020-11-15 Thread Craig Ringer
On Sun, Nov 15, 2020 at 1:49 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Fri, Nov 13, 2020 at 10:42 AM Bruce Momjian wrote: > >> I think the big problem, and I have seen this repeatedly, is showing up >> with a patch without discussing whether people actually want the >>

Re: Add docs stub for recovery.conf

2020-11-12 Thread Craig Ringer
On Fri, Nov 13, 2020 at 11:50 AM Bruce Momjian wrote: > > So you are saying you don't think you are getting sufficient thought > > into your proposal, and getting just a reflex? Just because we don't > > agree with you don't mean we didn't think about it. In fact, we have > > thought about it

Re: Add docs stub for recovery.conf

2020-11-12 Thread Craig Ringer
On Fri, Nov 13, 2020 at 11:31 AM Craig Ringer wrote: > > Can anyone tell me why the solution I proposed is not acceptable, and why > we have to invent a different one instead? The website redirect is good > and all, but doesn't really solve the problem, and I still don't know &g

Re: Add docs stub for recovery.conf

2020-11-12 Thread Craig Ringer
On Thu, Nov 12, 2020 at 11:25 PM Stephen Frost wrote: > > > Now, the pgweb feature that Jonathan wrote recently might actually be > > exactly what we need to fix that, and to address the issue with > > recovery config documentation that Craig raises. > > After chatting with Jonathan about this

Re: Add docs stub for recovery.conf

2020-11-11 Thread Craig Ringer
On Thu, Nov 12, 2020 at 4:01 AM Bruce Momjian wrote: > On Wed, Nov 11, 2020 at 08:59:40PM +0100, Daniel Gustafsson wrote: > > > On 11 Nov 2020, at 20:44, Bruce Momjian wrote: > > > On Tue, Nov 10, 2020 at 01:38:14PM +0800, Craig Ringer wrote: > > > > >

Re: Add docs stub for recovery.conf

2020-11-11 Thread Craig Ringer
On Thu, Nov 12, 2020 at 3:44 AM Bruce Momjian wrote: > On Tue, Nov 10, 2020 at 01:38:14PM +0800, Craig Ringer wrote: > > Hi all > > > > I noticed that when recovery.conf was removed in 2dedf4d9a8 (yay!) the > docs for > > it were removed completely as

Re: PATCH: Report libpq version and configuration

2020-11-10 Thread Craig Ringer
On Tue, Nov 10, 2020 at 2:22 PM Craig Ringer wrote: > > The main things I'd really like to get in place are a way to get the > version as an ELF data symbol, and a simple way to ID the binary. > > So the minimal change would be to declare: > > const char LIBPQ_VERSION_S

Re: PATCH: Report libpq version and configuration

2020-11-09 Thread Craig Ringer
On Tue, Nov 10, 2020 at 12:33 AM Tom Lane wrote: > Alvaro Herrera writes: > > Well, if we can make it run in more systems than just Linux, then it > > seems worth having. The submitted patch seems a little bit on the > > naughty side. > > I agree that the facility seems possibly useful, as

Add docs stub for recovery.conf

2020-11-09 Thread Craig Ringer
ing them. That'll help prevent us from breaking links on the wider web, in 3rd party documentation, etc. From 9d6db17c8bff093c08a90129d0ba181554b5d3e2 Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Tue, 10 Nov 2020 10:51:20 +0800 Subject: [PATCH] Link the old recovery.conf docs chapter to the ne

[PATCH] TAP test showing that pg_replication_slot_advance() works on standby

2020-11-09 Thread Craig Ringer
searchers can find it at least. From 218c607b64db493b975bb398528d6a952beeb32f Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Tue, 10 Nov 2020 13:06:41 +0800 Subject: [PATCH] Extend TAP test for pg_replication_slot_advance() to cover standby Show that pg_replication_slot_advance() works on a stan

Re: Internal key management system

2020-10-29 Thread Craig Ringer
On Thu, Oct 29, 2020 at 1:22 AM Stephen Frost wrote: > > Most importantly - I don't think the SQL key adds anything really > > crucial that we cannot do at the SQL level with an extension. An > > extension "pg_wrap" could provide pg_wrap() and pg_unwrap() already, > > using a single master key

Re: Internal key management system

2020-10-28 Thread Craig Ringer
On Wed, Oct 28, 2020 at 12:02 PM Craig Ringer wrote: > On Wed, Oct 28, 2020 at 9:43 AM Bruce Momjian wrote: > > > >> I don't know much about how to hook into that stuff so if you have an >> idea, I am all ears. > > > Yeah, I have a reasonable idea. The main thi

Re: Internal key management system

2020-10-27 Thread Craig Ringer
On Wed, Oct 28, 2020 at 9:43 AM Bruce Momjian wrote: > > I don't know much about how to hook into that stuff so if you have an > idea, I am all ears. Yeah, I have a reasonable idea. The main thing will be to re-read the patch and put it into more concrete terms, which I'll try to find time for

Re: Internal key management system

2020-10-27 Thread Craig Ringer
On Tue, 27 Oct 2020, 19:15 Bruce Momjian, wrote: > We could implement a 'command:' prefix now, and maybe > a 'pass:' one, and allow other methods like 'pkcs11' later. > We don't need to do anything except provide a way to tell OpenSSL where to get the KEK from, for situations where having Pg

Re: Internal key management system

2020-10-27 Thread Craig Ringer
On Mon, Oct 26, 2020 at 11:02 PM Stephen Frost wrote: TL;DR: * Important to check that key rotation is possible on a replica, i.e. primary and standby can have different cluster passphrase and KEK encrypting the same WAL and heap keys; * with a HSM we can't read the key out, so a pluggable KEK

Re: PATCH: Report libpq version and configuration

2020-10-26 Thread Craig Ringer
On Tue, Oct 27, 2020 at 12:56 AM Tom Lane wrote: > > Alvaro Herrera writes: > > On 2020-Oct-26, Craig Ringer wrote: > >> also adds PQlibInfoPrint() which dumps PQlibInfo() keys/values to stdout. > > > Sounds useful. I'd have PQlibInfoPrint(FILE *) instead, so you ca

Re: PATCH: Report libpq version and configuration

2020-10-26 Thread Craig Ringer
On Tue, Oct 27, 2020 at 12:41 AM Alvaro Herrera wrote: > On 2020-Oct-26, Craig Ringer wrote: > > > Patch 0001 adds PQlibInfo(), which returns an array of key/value > > description items reporting on configuration like the full version > string, > > SSL support, gs

Re: Internal key management system

2020-10-26 Thread Craig Ringer
On Mon, Oct 19, 2020 at 11:16 AM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: The patch introduces only key management infrastructure but with no > key. Currently, there is no interface to dynamically add a new > encryption key. I'm a bit confused by the exact intent and use cases

PATCH: Report libpq version and configuration

2020-10-26 Thread Craig Ringer
/lib64/ccache/gcc -E' USE_SSL: 0 ENABLE_GSS: 0 ENABLE_THREAD_SAFETY: 1 HAVE_UNIX_SOCKETS: 1 DEFAULT_PGSOCKET_DIR: /tmp DEF_PGPORT: 5432 From a4741725a9596b25a9b7308b6a47e85b889d4557 Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Mon, 26 Oct 2020 20:43:24 +0800 Subject: [PATCH v1 2/3] Add libpq

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-10-22 Thread Craig Ringer
On Wed, Oct 7, 2020 at 8:39 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Wed, Oct 7, 2020 at 8:00 AM Bharath Rupireddy > wrote: > > > > On Tue, Oct 6, 2020 at 11:41 AM Bharath Rupireddy > > wrote: > > > > > > On Tue, Oct 6, 2020 at 11:20 AM Fujii Masao < >

Re: Make procedure OUT parameters work with JDBC

2020-10-19 Thread Craig Ringer
On Mon, 19 Oct 2020, 19:16 Andrew Dunstan, wrote: > > On 10/19/20 5:19 AM, Peter Eisentraut wrote: > > A follow-up to the recently added support for OUT parameters for > > procedures. The JDBC driver sends OUT parameters with type void. > > This makes sense when calling a function, so that the

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2020-10-15 Thread Craig Ringer
On Fri, 16 Oct 2020, 09:00 Michael Paquier, wrote: > On Thu, Oct 15, 2020 at 01:06:54PM -0400, Tom Lane wrote: > > Other than src/test/modules/brin, the ISOLATION users don't look > > much like real extensions (rather than test scaffolding), either. > > If you discount test scaffolding modules

Re: Add primary keys to system catalogs

2020-10-06 Thread Craig Ringer
On Sun, 4 Oct 2020, 01:32 John Naylor, wrote: > > On Sat, Oct 3, 2020 at 9:27 AM Craig Ringer > wrote: > > So a big +1 from me for the idea. Especially if we ensure psql > recognises when the relation 'oid' attribute has a declared PK and includes > it in the column l

Re: Add primary keys to system catalogs

2020-10-03 Thread Craig Ringer
On Sat, 3 Oct 2020, 14:40 Peter Eisentraut, < peter.eisentr...@2ndquadrant.com> wrote: > I saw someone ask once for a schema diagram of the system catalogs. > Things like that have occasionally been produced manually, but they are > not regularly updated. That made me wonder, why can't we add

Re: Improving connection scalability: GetSnapshotData()

2020-10-01 Thread Craig Ringer
s anybody have a good > idea for how to best, and without undue effort, to integrate this into > PostgresNode.pm? I don't really have a great idea, so I think I'd leave > it with a local helper in the new test? 2ndQ has some infra for that and various other TAP enhancements that I'd like to try to

PoC patch: expose TCP socket stats for walsenders

2020-09-30 Thread Craig Ringer
for inspection. Or have monitoring tools poll it, that sort of thing. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise From ddfba877e3c386598e3790b674136717fc0b0321 Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Tue, 29 Sep 2020

pg_proc.dat "proargmodes is not a 1-D char array"

2020-09-29 Thread Craig Ringer
IMS(arr)[0] != numargs || in funcapi.c. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2020-09-29 Thread Craig Ringer
On Tue, 29 Sep 2020 at 22:09, Alvaro Herrera wrote: > On 2018-Jun-01, Craig Ringer wrote: > > > On 28 May 2018 at 15:06, Craig Ringer wrote: > > > > > Per topic, the Pg makefiles install pg_regress (for use by extensions) > and > > > htey install the

[PATCH] Runtime control of CLOBBER_CACHE_ALWAYS

2020-09-25 Thread Craig Ringer
. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise From 372defde443d178fb4a9c8cf4092dea7debf72ea Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Tue, 22 Sep 2020 09:51:00 +0800 Subject: [PATCH v1] Add runtime control over

Proposals for making it easier to write correct bgworkers

2020-09-09 Thread Craig Ringer
and end */ Assert(CurrentMemoryContext == txn->save_mctx); } (void) MemoryContextSwitchTo(txn->save_mctx); } -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: [PATCH] Detect escape of ErrorContextCallback stack pointers (and from PG_TRY() )

2020-09-09 Thread Craig Ringer
oor-man's backtrace through each PG_TRY() at the cost of one sizeof(void) per PG_TRY() + some constdata storage, but that seems like a solution looking for a problem. Also, AFAICS it's actually harmless, if ugly, to return from PG_CATCH(). It should still be discouraged. -- Craig Ringer

Re: [PATCH] Detect escape of ErrorContextCallback stack pointers (and from PG_TRY() )

2020-09-07 Thread Craig Ringer
On Fri, 4 Sep 2020 at 14:55, Craig Ringer wrote: > Using a test program: > > return_stack_escape.c:14:3: warning: Address of stack memory associated > with local variable 'g' is still referred to by the global variable > 'guard_ptr' upon returning to the caller. This wil

Re: [PATCH] Detect escape of ErrorContextCallback stack pointers (and from PG_TRY() )

2020-09-04 Thread Craig Ringer
On Fri, 4 Sep 2020 at 14:13, Craig Ringer wrote: > > I actually had a pretty good look around for static analysis options to > see if I could find anything that might help us out before I landed up with > this approach. > Apparently not good enough. https://clang.llvm.or

Re: [PATCH] Detect escape of ErrorContextCallback stack pointers (and from PG_TRY() )

2020-09-04 Thread Craig Ringer
On Thu, 3 Sep 2020 at 22:28, Tom Lane wrote: > Craig Ringer writes: > > The attached patch series adds support for detecting coding errors where > a > > stack-allocated ErrorContextCallback is not popped from the > > error_context_stack before the variable leaves sc

[PATCH] Detect escape of ErrorContextCallback stack pointers (and from PG_TRY() )

2020-09-03 Thread Craig Ringer
these changes, but it might make sense to adopt it at the same time if we're changing usage across the tree anyway. A further patch (7) switches over to using __has_attribute instead of explicit compiler version checks. Again, optional, more of a RFC. -- Craig Ringer http://www.2n

Re: Two fsync related performance issues?

2020-05-26 Thread Craig Ringer
On Tue, 12 May 2020, 08:42 Paul Guo, wrote: > Hello hackers, > > 1. StartupXLOG() does fsync on the whole data directory early in the crash > recovery. I'm wondering if we could skip some directories (at least the > pg_log/, table directories) since wal, etc could ensure consistency. Here > is

Re: Remove page-read callback from XLogReaderState.

2020-05-26 Thread Craig Ringer
On Tue, 26 May 2020, 15:40 Kyotaro Horiguchi, wrote: > > This patch removes all the three callbacks (open/close/page_read) in > XL_ROUTINE from XLogReaderState. It only has "cleanup" callback > instead. > I actually have a use in mind for these callbacks - to support reading WAL for logical

Re: what can go in root.crt ?

2020-05-26 Thread Craig Ringer
te certificate of WE ISSUE TO ORGS LIKE YOURS.COM . Assuming the client will accept it; not all clients allow you to configure "certificates I trust to sign peers" separately to "certificates that sign my trusted roots". Because really, in security terms that's nonsensical. --

Re: [PATCH] Fix install-tests target for vpath builds

2020-05-25 Thread Craig Ringer
On Thu, 23 Apr 2020 at 12:55, Craig Ringer wrote: > Patch 0001 fixes this issue with vpath postgres builds: > > $ make -C src/test/regress install-tests > /usr/bin/install: cannot create regular file > 'PGPREFIX/lib/postgresql/regress/PGPREFIX/src/test/regress/expe

Re: [PATCH] Skip llvm bytecode generation if LLVM is missing

2020-04-22 Thread Craig Ringer
On Thu, 19 Mar 2020 at 18:47, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-03-15 02:28, Craig Ringer wrote: > > On Fri, 13 Mar 2020 at 15:04, Andres Freund > <mailto:and...@anarazel.de>> wrote: > > > > On 2020-0

[PATCH] Fix install-tests target for vpath builds

2020-04-22 Thread Craig Ringer
. This lets extensions that want to define a 'check' target do so without having make complain at them about redefined targets. This patch is independent of the others and can apply on master directly. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-15 Thread Craig Ringer
On Thu, 16 Apr 2020 at 13:23, Craig Ringer wrote: > > Just discarding the prepared xacts is not the answer though. > ... however, I have wondered a few times about making vacuum smarter about cases where the xmin is held down by prepared xacts or by replication slots. If we cou

Re: Do we need to handle orphaned prepared transactions in the server?

2020-04-15 Thread Craig Ringer
having to watch the logs or query obscure state to understand what the system is doing and why bloat is being created by abandoned prepared xacts. Just discarding the prepared xacts is not the answer though. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: Include sequence relation support in logical replication

2020-04-15 Thread Craig Ringer
On Thu, 16 Apr 2020 at 07:44, Andres Freund wrote: > > > I would like to ask if you have some suggestions or ideas that can make > subscriber receives the current value without the need to > > > > disabling the 32 increment behavior? > > It simply shouldn't expect that to be the case. What do

Re: [PATCH] Support built control file in PGXS VPATH builds

2020-04-08 Thread Craig Ringer
On Mon, 30 Mar 2020 at 11:50, Craig Ringer wrote: > > > On Mon, 9 Mar 2020, 17:27 Peter Eisentraut, < > peter.eisentr...@2ndquadrant.com> wrote: > >> On 2020-02-07 04:14, Craig Ringer wrote: >> > The attached patch fixes this by having PGXS resolve >&

Re: Proposal: Expose oldest xmin as SQL function for monitoring

2020-04-01 Thread Craig Ringer
doesn't help you > find *where* it is. Maybe what we need is a view showing all of > these potential sources of an old xmin. Strongly agree. https://www.postgresql.org/message-id/camsr+ygss6jbhmehbxqmdc1xj7sobdsq62ywaekohn-kbqy...@mail.gmail.com I was aiming to write such a view, bu

Re: Missing errcode() in ereport

2020-03-29 Thread Craig Ringer
On Fri, 20 Mar 2020, 01:59 Andres Freund, wrote: > Hi, > > On 2020-03-17 10:09:18 -0400, Tom Lane wrote: > > We might want to spend some effort thinking how to find or prevent > > additional bugs of the same ilk ... > > Yea, that'd be good. Trying to help people new to postgres write their >

Re: [PATCH] Support built control file in PGXS VPATH builds

2020-03-29 Thread Craig Ringer
On Mon, 9 Mar 2020, 17:27 Peter Eisentraut, < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-02-07 04:14, Craig Ringer wrote: > > The attached patch fixes this by having PGXS resolve > > $(EXTENSION).control along the VPATH. > > Simpler patch: > > diff --

Re: [PATCH] Skip llvm bytecode generation if LLVM is missing

2020-03-14 Thread Craig Ringer
On Fri, 13 Mar 2020 at 15:04, Andres Freund wrote: > On 2020-03-13 14:08:12 +0800, Craig Ringer wrote: > > The alternative would be to detect a missing clang and emit a much more > > informative error than the current one that explicitly suggests retrying > > with >

Re: logical replication empty transactions

2020-03-13 Thread Craig Ringer
On Tue, 10 Mar 2020 at 02:30, Andres Freund wrote: > Hi, > > On 2020-03-06 13:53:02 +0800, Craig Ringer wrote: > > On Mon, 2 Mar 2020 at 19:26, Amit Kapila > wrote: > > > > > One thing that is not clear to me is how will we advance restart_lsn > > > i

Re: [PATCH] Skip llvm bytecode generation if LLVM is missing

2020-03-13 Thread Craig Ringer
=no or setting with_llvm=no in the environment. The whole thing is a mess caused by this "enterprise-y" repository split between core and "extras" and I'm rather frustrated by the whole thing, but the current situation isn't much fun for users. -- Craig Ringer http://w

Re: [PATCH] Skip llvm bytecode generation if LLVM is missing

2020-03-12 Thread Craig Ringer
On Thu, 12 Mar 2020 at 03:43, Andres Freund wrote: > On 2020-03-11 11:25:28 +0800, Craig Ringer wrote: > > I propose that per the attached patch PGXS should simply skip adding > > the automatic dependency for .bc files if clang cannot be found. > > Extensions may still

Re: [PATCH] Skip llvm bytecode generation if LLVM is missing

2020-03-11 Thread Craig Ringer
On Wed, 11 Mar 2020 at 13:47, Pavel Stehule wrote: > > st 11. 3. 2020 v 6:43 odesílatel Julien Rouhaud napsal: >> >> Le mer. 11 mars 2020 à 05:28, Laurenz Albe a >> écrit : >>> >>> On Wed, 2020-03-11 at 11:25 +0800, Craig Ringer wrote: >>>

Re: custom postgres launcher for tests

2020-03-11 Thread Craig Ringer
On Fri, 21 Feb 2020 at 17:05, Ivan N. Taranov wrote: > > On Fri, Feb 21, 2020 at 4:49 AM Craig Ringer wrote: > > > I thought I saw a related patch to this that proposed to add a pg_ctl > > argument. Was that you too? I can't find it at the moment. I've had it on my TOD

Re: Proposal: PqSendBuffer removal

2020-03-10 Thread Craig Ringer
faifaiOn Sat, 7 Mar 2020 at 02:45, Andres Freund wrote: > > Hi, > > On March 5, 2020 1:23:21 PM PST, Aleksei Ivanov wrote: > >Thank you for your reply! > > > >Yes, you are right there will be a separate call to send the data, but > >is > >copying data each time more costly operation than just

[PATCH] Skip llvm bytecode generation if LLVM is missing

2020-03-10 Thread Craig Ringer
, so it all works. But I don't think it's worth the hassle and I'd rather just skip automatic bitcode generation if we don't find clang. See also yum packagers report at https://www.postgresql.org/message-id/CAMsr+YHokx0rWLV561z3=gai6cm4yjekgclkqmdwqstejvy...@mail.gmail.com . -- Craig Ringer

Re: logical replication empty transactions

2020-03-05 Thread Craig Ringer
onse from the replica(s), which delays synchronous replication. You need to send empty txns when synchronous replication is enabled, or instead poke the walsender to force immediate feedback with reply requested. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL

Re: Proposal: PqSendBuffer removal

2020-03-05 Thread Craig Ringer
o we had a way to recover after interrupted sending of a partial message... -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: Add PGURI env var for passing connection string to psql in Docker

2020-02-20 Thread Craig Ringer
ntainerised process(es) from outside the container. Much more likely than being able to peek at a file, anyway. Then again, Docker relies on dropping capabilities and likes to run as root-that-isn't-root-except-when-it's-root, which doesn't thrill me when it comes to security. At al

Re: custom postgres launcher for tests

2020-02-20 Thread Craig Ringer
t getting a pg_ctl command line or the like into all the places where we'd want it deep inside TAP tests etc is just too much hassle otherwise. I haven't reviewed the code yet, but for the idea a strong +1000 or so. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL

Re: Internal key management system

2020-02-18 Thread Craig Ringer
t say "that's your problem" to people in those places. I wholly support this approach. Pretty much everything is illegal somewhere. Patents are pain enough already. (Apologies for thread-breaking reply, this is not from my usually-subscribed account. I do not speak in any way for my employer on this matter.) -- Craig Ringer

Re: PL/Python - lifetime of variables?

2020-02-18 Thread Craig Ringer
velopment practice. Instead, use a "with" block, or a try/finally block, and do explicit cleanup for external resources. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: Standards compliance of SET ROLE / SET SESSION AUTHORIZATION

2020-02-16 Thread Craig Ringer
. * When the client releases the session, pooler does a protocol-level RESET SESSION AUTHORIZATION to the pooler user, supplying the reset cookie it gave at S-S-A time. > > regards, tom lane > > -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: [Proposal] Add accumulated statistics for wait event

2020-02-11 Thread Craig Ringer
like perf, bpftrace and systemtap. That way we have a zero-overhead-when-unused option that can also be used to aggregate the information per-query, per-user, etc. (I really need to add a bunch more tracepoints to make this easier...) -- Craig Ringer http://www.2ndQua

Re: POC: GUC option for skipping shared buffers in core dumps

2020-02-10 Thread Craig Ringer
repro crash on a production system I really don't want 20GB coredumps... Please, please apply. Please backpatch, if you can possibly stand to do so. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: Postgres 32 bits client compilation fail. Win32 bits client is supported?

2020-02-10 Thread Craig Ringer
ains versions on a machine. I now maintain isolated VMs with exactly one MSVC version on each to address the amazing level of breakage and incompatibility that MS's various toolchains seem to deliver. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: Is custom MemoryContext prohibited?

2020-02-10 Thread Craig Ringer
On Mon, 10 Feb 2020 at 21:19, Kohei KaiGai wrote: > > 2020年2月10日(月) 13:53 Craig Ringer : > > > > On Thu, 6 Feb 2020 at 11:09, Andres Freund wrote: > > > > > I wasn't advocating for making plannodes.h etc frontend usable. I think > > > that's a fairly dif

Re: WAL usage calculation patch

2020-02-09 Thread Craig Ringer
sort of thing. (I aim to find time to submit a patch for that.) I haven't yet reviewed the patch. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

Re: Is custom MemoryContext prohibited?

2020-02-09 Thread Craig Ringer
without modifying all the code that knows how to work with the parent types. MemoryContext is one example of this, Node is another. The underlying issue is similar. Being able to do this is something I'm much more interested in being able to do for plan and parse nodes etc than for MemoryContext tbh

Re: Postgres 32 bits client compilation fail. Win32 bits client is supported?

2020-02-09 Thread Craig Ringer
ll Postgres continue to support 32-bit client? > > > > I am not aware of any discussion related to stopping the support of > 32-bit client. Buildfarm member whelk reports for 32-bit Windows. https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=whelk=HEAD https://buildfarm

[PATCH] Support built control file in PGXS VPATH builds

2020-02-06 Thread Craig Ringer
have a few other cleanup/fixup patches in the pipe for PGXS and Makefile.global but I have to tidy them up a bit first. One to eliminate undefined variables use, another to allow vpath directives to be used instead of the big VPATH variable hammer. Keep an eye out. -- Craig Ringer

Re: Unix-domain socket support on Windows

2020-01-30 Thread Craig Ringer
t'd take a Windows-competent person to review. But > the patch as it stands isn't that. > > The code looks fine (and a big +1 for not having knowledge of > DEFAULT_PGSOCKET_DIR wired into UNIXSOCK_PATH). I wonder though > whether any user-facing documentation needs to be adjusted.

Re: MSVC installs too much stuff?

2020-01-30 Thread Craig Ringer
't have PGXS. It probably doesn't make sense to install them to the pgxs dir. So putting them in bin is pretty reasonable. -- Craig Ringer http://www.2ndQuadrant.com/ 2ndQuadrant - PostgreSQL Solutions for the Enterprise

<    1   2   3   4   5   6   >