Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Kyotaro Horiguchi
At Fri, 5 Mar 2021 10:07:06 +0500, Andrey Borodin wrote in > Maybe it's offtopic here, but anyway... > While working on "lz4 for FPIs" I've noticed that this test fails with > wal_compression = on. > I did not investigate the case at that moment, but I think that it would be > good to run

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Kyotaro Horiguchi
At Thu, 04 Mar 2021 23:40:34 -0500, Tom Lane wrote in > BTW, I tried simply removing the "allows_streaming" option from > the test, and it failed ten times out of ten tries for me. > So Andres is right that that makes it pretty reproducible in > a stock build. The difference comes from the

Re: Which PG version does CVE-2021-20229 affected?

2021-03-04 Thread Michael Paquier
On Fri, Mar 05, 2021 at 12:32:43AM -0700, bchen90 wrote: > NVD link: > > https://nvd.nist.gov/vuln/detail/CVE-2021-20229#vulnCurrentDescriptionTitle This link includes incorrect information. CVE-2021-20229 is only a problem in 13.0 and 13.1, fixed in 13.2. Please see for example here:

Re: [PATCH] pgbench: Bug fix for the -d option

2021-03-04 Thread Michael Paquier
On Fri, Mar 05, 2021 at 01:30:11PM +0900, Fujii Masao wrote: > if ((env = getenv("PGDATABASE")) != NULL && *env != '\0') > dbName = env; > - else if (login != NULL && *login != '\0') > - dbName = login; > + else if

Which PG version does CVE-2021-20229 affected?

2021-03-04 Thread bchen90
Hi, all Recently, I retrieved CVE-2021-20229 on the NVD website which describes the affected PG version are "before 13.2, before 12.6, before 11.11, before 10.16, before 9.6.21 and before 9.5.25", but we I look the official website of PG and look the git commit log, I found only 13 version is

Re: Disallow SSL compression?

2021-03-04 Thread Michael Paquier
On Thu, Mar 04, 2021 at 11:52:56PM +0100, Daniel Gustafsson wrote: > The attached version takes a step further and removes sslcompression from > pg_conn and just eats the value as there is no use in setting a dummy alue. > It > also removes compression from PgBackendSSLStatus and

Re: n_mod_since_analyze isn't reset at table truncation

2021-03-04 Thread Julien Rouhaud
On Fri, Mar 05, 2021 at 03:31:33PM +0900, Fujii Masao wrote: > > On 2021/03/04 12:40, Julien Rouhaud wrote: > > > In that case, conversely, we want to trigger autoanalyze ASAP because the > > > contents in the table was changed very much? > > > > We might want, but wouldn't keeping the current

RE: Avoid CommandCounterIncrement in RI trigger when INSERT INTO referencing table

2021-03-04 Thread houzj.f...@fujitsu.com
> > > I'm worried about having this dependency in RI check, because the > > > planner > > may allow parallel INSERT in these cases in the future. > > > > If we support parallel insert that can have other modifications in the > > future, I think we will also be able to share or increment command

RE: Avoid CommandCounterIncrement in RI trigger when INSERT INTO referencing table

2021-03-04 Thread tsunakawa.ta...@fujitsu.com
From: Hou, Zhijie/侯 志杰 > From the wiki[1], CCI is to let statements can not see the rows they modify. > > Here is an example of the case 1): > (Note table referenced and referencing are both empty) > - > postgres=# with cte as (insert into referenced values(1)) insert into > referencing

Re: n_mod_since_analyze isn't reset at table truncation

2021-03-04 Thread Fujii Masao
On 2021/03/04 12:40, Julien Rouhaud wrote: On Thu, Mar 04, 2021 at 12:21:14PM +0900, Fujii Masao wrote: On 2021/03/04 11:24, Julien Rouhaud wrote: On Thu, Mar 04, 2021 at 10:35:19AM +0900, Masahiko Sawada wrote: While reviewing "autoanalyze on partitioned table" patch, I realized that

RE: libpq debug log

2021-03-04 Thread tsunakawa.ta...@fujitsu.com
From: Tom Lane > But I think passing the message start address explicitly might be better than > having it understand the buffering behavior in enough detail to know where to > find the message. Part of the point here > (IMO) is to decouple the tracing logic from the core libpq logic, in hopes

Re: Shared memory size computation oversight?

2021-03-04 Thread Julien Rouhaud
On Thu, Mar 04, 2021 at 12:53:54PM -0500, Tom Lane wrote: > My conclusion is that we don't need to do anything, indeed the proposed > changes will probably just lead to overestimation. > > It's certainly possible that there's something amiss somewhere. These > numbers were all taken with

Re: Improvements and additions to COPY progress reporting

2021-03-04 Thread Michael Paquier
On Thu, Mar 04, 2021 at 05:45:50PM +0100, Matthias van de Meent wrote: > Sure, I'm convinced. PFA the patchset with this change applied. 0002 looks fine to me, and in line with the discussion, so applied. -- Michael signature.asc Description: PGP signature

Re: SQL-standard function body

2021-03-04 Thread Jaime Casanova
On Tue, Mar 2, 2021 at 12:13 PM Peter Eisentraut wrote: > > On 11.02.21 09:02, Peter Eisentraut wrote: > >> Here is an updated patch to get it building again. > > > > Another updated patch to get things building again. I've also fixed the > > last TODO I had in there in qualifying function

Re: libpq debug log

2021-03-04 Thread Tom Lane
"tsunakawa.ta...@fujitsu.com" writes: > Oops, the logging facility needs the destination (conn->pfdebug). So, it > will be: > void pqLogMessage(PGconn *conn, bool toServer); Right, and it'd need the debug flags too, so +1 for just passing the PGconn instead of handing those over

RE: libpq debug log

2021-03-04 Thread tsunakawa.ta...@fujitsu.com
From: tsunakawa.ta...@fujitsu.com > I understood that the former is pqParseInput3() and the latter is > pqPutMsgEnd(). They call the logging function wne conn->pfdebug is not > NULL. Its signature is like this (that will be defined in libpq-trace.c): > > void pqLogMessage(const char

RE: libpq debug log

2021-03-04 Thread tsunakawa.ta...@fujitsu.com
Tom-san, Alvaro-san, From: Tom Lane > I took a quick look through the v22 patch, and TBH I don't like much of > anything > at all about the proposed architecture. It's retained most of the flavor of > the > way it was done before, which was a hangover from the old process-on-the-fly >

Re: Get memory contexts of an arbitrary backend process

2021-03-04 Thread Fujii Masao
On 2021/03/04 18:32, torikoshia wrote: On 2021-01-14 19:11, torikoshia wrote: Since pg_get_target_backend_memory_contexts() waits to dump memory and it could lead dead lock as below.   - session1   BEGIN; TRUNCATE t;   - session2   BEGIN; TRUNCATE t; -- wait   - session1   SELECT * FROM

Re: pgbench: option delaying queries till connections establishment?

2021-03-04 Thread Thomas Munro
On Thu, Mar 4, 2021 at 10:44 PM Thomas Munro wrote: > v10-0002-pgbench-Refactor-the-way-we-do-thread-portabilit.patch Here's a better version of that part. I don't yet know if it actually works on Windows... From 3aa63dfc086ab1f687ed668091a6bda8bf270fa7 Mon Sep 17 00:00:00 2001 From: Thomas

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Andrey Borodin
> 5 марта 2021 г., в 08:32, Tom Lane написал(а): > > Kyotaro Horiguchi writes: >> I noticed that 011_crash_recovery.pl intermittently (that being said, >> one out of three or so on my environment) fails in the second test. > > Hmmm ... what environment is that? This test script hasn't

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Kyotaro Horiguchi
At Fri, 05 Mar 2021 13:21:48 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 05 Mar 2021 13:13:04 +0900 (JST), Kyotaro Horiguchi > wrote in > > At Thu, 04 Mar 2021 23:02:09 -0500, Tom Lane wrote in > > > Having said that, it's still true that this test has been stable in > > > the

Re: n_mod_since_analyze isn't reset at table truncation

2021-03-04 Thread Masahiko Sawada
On Thu, Mar 4, 2021 at 12:39 PM Julien Rouhaud wrote: > > On Thu, Mar 04, 2021 at 12:21:14PM +0900, Fujii Masao wrote: > > > > > > On 2021/03/04 11:24, Julien Rouhaud wrote: > > > On Thu, Mar 04, 2021 at 10:35:19AM +0900, Masahiko Sawada wrote: > > > > > > > > While reviewing "autoanalyze on

Re: n_mod_since_analyze isn't reset at table truncation

2021-03-04 Thread Masahiko Sawada
On Thu, Mar 4, 2021 at 11:23 AM Julien Rouhaud wrote: > > On Thu, Mar 04, 2021 at 10:35:19AM +0900, Masahiko Sawada wrote: > > > > While reviewing "autoanalyze on partitioned table" patch, I realized > > that pg_stat_xxx_tables.n_mod_since_analyze is not reset at TRUNCATE. > > On the other hand,

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Tom Lane
Thomas Munro writes: > On Fri, Mar 5, 2021 at 5:10 PM Tom Lane wrote: >> Alternatively, maybe we can salvage the function's usefulness by making it >> flush WAL before returning? > To make pg_xact_status()'s result reliable, don't you need to make > pg_current_xact_id() flush? In other words,

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Thomas Munro
On Fri, Mar 5, 2021 at 5:10 PM Tom Lane wrote: > I wrote: > > I'd be kind of inclined to remove this test script altogether, on the > > grounds that it's wasting cycles on a function that doesn't really > > do what is claimed (and we should remove the documentation claim, too). > > Alternatively,

Re: [PATCH] pgbench: Bug fix for the -d option

2021-03-04 Thread Fujii Masao
On 2021/03/05 11:26, miyake_kouta wrote: 2021-03-04 21:11, Michael Paquier wrote: Like any other src/bin/ facilities, let's instead remove all the getenv() calls at the beginning of pgbench.c and let's libpq handle those environment variables if the parameters are NULL (aka in the case of no

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Kyotaro Horiguchi
At Fri, 05 Mar 2021 13:13:04 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 04 Mar 2021 23:02:09 -0500, Tom Lane wrote in > > Peter Geoghegan writes: > > > On Thu, Mar 4, 2021 at 7:32 PM Tom Lane wrote: > > >> Hmmm ... what environment is that? This test script hasn't changed > > >>

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Kyotaro Horiguchi
At Thu, 04 Mar 2021 23:02:09 -0500, Tom Lane wrote in > Peter Geoghegan writes: > > On Thu, Mar 4, 2021 at 7:32 PM Tom Lane wrote: > >> Hmmm ... what environment is that? This test script hasn't changed > >> meaningfully in several years, and we have not seen any real issues > >> with it up

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Tom Lane
I wrote: > I'd be kind of inclined to remove this test script altogether, on the > grounds that it's wasting cycles on a function that doesn't really > do what is claimed (and we should remove the documentation claim, too). Alternatively, maybe we can salvage the function's usefulness by making

RE: [PATCH] pgbench: improve \sleep meta command

2021-03-04 Thread kuroda.hay...@fujitsu.com
Dear Miyake-san, > I'm not sure but I think this is caused because `my_command->argv[2]` > becomes "foo". Sorry, I missed some lines in your patch. Please ignore this analysis. Best Regards, Hayato Kuroda FUJITSU LIMITED

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Tom Lane
Peter Geoghegan writes: > On Thu, Mar 4, 2021 at 7:32 PM Tom Lane wrote: >> Hmmm ... what environment is that? This test script hasn't changed >> meaningfully in several years, and we have not seen any real issues >> with it up to now. > Did you see this recent thread? >

Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW

2021-03-04 Thread Japin Li
On Thu, 04 Mar 2021 at 14:53, Bharath Rupireddy wrote: > Thanks! I will look forward for more review comments. > v4-0001-Rearrange-Refresh-Mat-View-Code.patch - +static Oid +get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid

RE: [PATCH] pgbench: improve \sleep meta command

2021-03-04 Thread kuroda.hay...@fujitsu.com
Dear Miyake-san, I agree your suggestions and I think this patch is basically good. I put some comments: * When the following line is input, the error message is not happy. I think output should be " \sleep command argument must be an integer...". \sleep foo -> pgbench: fatal: test.sql:5:

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-04 Thread Fujii Masao
On 2021/03/05 8:38, Masahiro Ikeda wrote: On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote: On 2021-03-03 16:30, Fujii Masao wrote: On 2021/03/03 14:33, Masahiro Ikeda wrote: On 2021-02-24 16:14, Fujii Masao

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Peter Geoghegan
On Thu, Mar 4, 2021 at 7:32 PM Tom Lane wrote: > Hmmm ... what environment is that? This test script hasn't changed > meaningfully in several years, and we have not seen any real issues > with it up to now. Did you see this recent thread?

Re: 011_crash_recovery.pl intermittently fails

2021-03-04 Thread Tom Lane
Kyotaro Horiguchi writes: > I noticed that 011_crash_recovery.pl intermittently (that being said, > one out of three or so on my environment) fails in the second test. Hmmm ... what environment is that? This test script hasn't changed meaningfully in several years, and we have not seen any real

Re: [PATCH] remove deprecated v8.2 containment operators

2021-03-04 Thread Justin Pryzby
On Thu, Mar 04, 2021 at 08:58:39PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > [ 0001-remove-deprecated-v8.2-containment-operators.patch ] > > I'm confused by why this patch is only dropping the operators' > opclass-membership links. Don't we want to actually DROP OPERATOR > too? Okay

Re: Track replica origin progress for Rollback Prepared

2021-03-04 Thread Amit Kapila
On Fri, Mar 5, 2021 at 7:11 AM Amit Kapila wrote: > > On Thu, Mar 4, 2021 at 8:03 PM Amit Kapila wrote: > > > > I have just checked via code coverage that we don't seem to have tests > for recovery of replication origin advance for commit [1], see > function xact_redo_commit. Similarly, a

Re: Corruption during WAL replay

2021-03-04 Thread Kyotaro Horiguchi
At Thu, 4 Mar 2021 22:37:23 +0500, Ibrar Ahmed wrote in > The regression is failing for this patch, do you mind look at that and send > the updated patch? > > https://api.cirrus-ci.com/v1/task/6313174510075904/logs/test.log > > ... > t/006_logical_decoding.pl ok >

011_crash_recovery.pl intermittently fails

2021-03-04 Thread Kyotaro Horiguchi
Hello. I noticed that 011_crash_recovery.pl intermittently (that being said, one out of three or so on my environment) fails in the second test. > t/011_crash_recovery.pl .. 2/3 > # Failed test 'new xid after restart is greater' > # at t/011_crash_recovery.pl line 56. > # '539' > #

Re: [PATCH] pgbench: Bug fix for the -d option

2021-03-04 Thread miyake_kouta
2021-03-04 21:11, Michael Paquier wrote: Like any other src/bin/ facilities, let's instead remove all the getenv() calls at the beginning of pgbench.c and let's libpq handle those environment variables if the parameters are NULL (aka in the case of no values given directly in the options). This

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-04 Thread Andy Fan
On Fri, Mar 5, 2021 at 12:00 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Thu, Feb 18, 2021 at 08:58:13PM +0800, Andy Fan wrote: > > Thanks for continuing work on this patch! > > > On Tue, Feb 16, 2021 at 12:01 PM David Rowley > wrote: > > > > > On Fri, 12 Feb 2021 at 15:18, Andy Fan

Re: [PATCH] remove deprecated v8.2 containment operators

2021-03-04 Thread Tom Lane
Justin Pryzby writes: > [ 0001-remove-deprecated-v8.2-containment-operators.patch ] I'm confused by why this patch is only dropping the operators' opclass-membership links. Don't we want to actually DROP OPERATOR too? Also, the patch seems to be trying to resurrect hstore--1.0--1.1.sql, which

RE: Refactor ECPGconnect and allow IPv6 connection

2021-03-04 Thread kuroda.hay...@fujitsu.com
Dear Hackers, I reviewed for myself and fixed something: * refactor parse_options(), same as conninfo_uri_parse_params() in libpq Skipping blanks is needed in this functions because ecpg precompiler add additional blanks between option parameters. I did not fix precompiler because of the

Re: make coverage-html would fail within build directory separate from source tree

2021-03-04 Thread walker
Thanks for your reminder, Tom. Before I understand VPATH well, I always thought "outside the source tree" means the build tree is not under source tree. Here because of VPATH build, which means build tree should be a subdirectory of source tree. according to this rule, I retry this scenario

[PATCH] pgbench: improve \sleep meta command

2021-03-04 Thread miyake_kouta
Hi. I created a patch to improve \sleep meta command in pgbench. There are two problems with the current pgbench implementation of \sleep. First, when the input is like "\sleep foo s" , this string will be treated as 0 through atoi function, and no error will be raised. Second, when the input

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-04 Thread Amit Kapila
On Fri, Mar 5, 2021 at 5:07 AM Greg Nancarrow wrote: > > On Thu, Mar 4, 2021 at 11:05 PM Amit Kapila wrote: > > > > On Thu, Mar 4, 2021 at 5:24 PM Greg Nancarrow wrote: > > > > > > On Thu, Mar 4, 2021 at 10:07 PM Amit Kapila > > > wrote: > > > > > > > > > > >Also, in > > > > standard_planner,

Re: Track replica origin progress for Rollback Prepared

2021-03-04 Thread Amit Kapila
On Thu, Mar 4, 2021 at 8:03 PM Amit Kapila wrote: > > Having said that, I think we use > replication origins to test it. For example: > > Create Table t1(c1 int); > > SELECT pg_replication_origin_create('regress'); > SELECT pg_replication_origin_session_setup('regress'); > SELECT

Re: Extensibility of the PostgreSQL wire protocol

2021-03-04 Thread Jan Wieck
On 3/4/21 7:38 PM, Hannu Krosing wrote: On Thu, Mar 4, 2021 at 9:55 PM Jan Wieck wrote: but the whole thing was developed that way from the beginning and it is working. I don't have a definitive date when that code will be presented. Kuntal or Prateek may be able to fill in more details. Are

Re: Add support for PROVE_FLAGS and PROVE_TESTS in MSVC scripts

2021-03-04 Thread Michael Paquier
On Thu, Mar 04, 2021 at 06:37:56PM +0100, Juan José Santamaría Flecha wrote: > LGTM. Thanks. I have tested more combinations of options, came back a bit to the documentation for buildenv.pl where copying the values from the docs would result in a warning, and applied it. -- Michael

Re: PoC/WIP: Extended statistics on expressions

2021-03-04 Thread Tomas Vondra
On 3/5/21 1:43 AM, Justin Pryzby wrote: > On Mon, Jan 04, 2021 at 09:45:24AM -0600, Justin Pryzby wrote: >> On Mon, Jan 04, 2021 at 03:34:08PM +, Dean Rasheed wrote: >>> * I'm not sure I understand the need for 0001. Wasn't there an earlier >>> version of this patch that just did it by

Re: PoC/WIP: Extended statistics on expressions

2021-03-04 Thread Justin Pryzby
On Mon, Jan 04, 2021 at 09:45:24AM -0600, Justin Pryzby wrote: > On Mon, Jan 04, 2021 at 03:34:08PM +, Dean Rasheed wrote: > > * I'm not sure I understand the need for 0001. Wasn't there an earlier > > version of this patch that just did it by re-populating the type > > array, but which still

Re: Extensibility of the PostgreSQL wire protocol

2021-03-04 Thread Hannu Krosing
On Thu, Mar 4, 2021 at 9:55 PM Jan Wieck wrote: > > Another possibility, and this is what is being used by the AWS team > implementing the TDS protocol for Babelfish, is to completely replace > the entire TCOP mainloop function PostgresMain(). I suspect this is the only reasonable way to do it

Re: macOS SIP, next try

2021-03-04 Thread Tom Lane
Peter Eisentraut writes: > On 01.03.21 15:44, Tom Lane wrote: >> Peter Eisentraut writes: >>> I have since learned that there is a way to disable only the part of SIP >>> that is relevant for us. This seems like a useful compromise, and it >>> appears that a number of other open-source projects

Re: PROXY protocol support

2021-03-04 Thread Álvaro Hernández
On 5/3/21 0:21, Jacob Champion wrote: > On Thu, 2021-03-04 at 21:45 +0100, Magnus Hagander wrote: >> On Thu, Mar 4, 2021 at 9:07 PM Jacob Champion wrote: >>> Idle thought I had while setting up a local test rig: Are there any >>> compelling cases for allowing PROXY packets to arrive over Unix

Re: WIP: BRIN multi-range indexes

2021-03-04 Thread Tomas Vondra
On 3/4/21 5:30 PM, John Naylor wrote: > > On Tue, Mar 2, 2021 at 7:32 PM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: >> Ummm, in brin_minmax_multi_distance_timetz()? I don't think timetz can >> be infinite, no? I think brin_minmax_multi_distance_date you meant? > > In

Re: PROXY protocol support

2021-03-04 Thread Hannu Krosing
The current proposal seems to miss the case of transaction pooling (and statement pooling) where the same established connection multiplexes transactions / statements from multiple remote clients. What we would need for that case would be a functionl pg_set_remote_client_address( be_key,

Re: PITR promote bug: Checkpointer writes to older timeline

2021-03-04 Thread Soumyadeep Chakraborty
Hey all, I took a stab at a quick and dirty TAP test (my first ever). So it can probably be improved a lot. Please take a look. On Thu, Mar 04, 2021 at 10:28:31AM +0900, Kyotaro Horiguchi wrote: > 2. Restore ThisTimeLineID after calling XLogReadRecord() in the > *caller* side. This is what

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-04 Thread Masahiro Ikeda
On 2021-03-05 01:02, Fujii Masao wrote: On 2021/03/04 16:14, Masahiro Ikeda wrote: On 2021-03-03 20:27, Masahiro Ikeda wrote: On 2021-03-03 16:30, Fujii Masao wrote: On 2021/03/03 14:33, Masahiro Ikeda wrote: On 2021-02-24 16:14, Fujii Masao wrote: On 2021/02/15 11:59, Masahiro Ikeda wrote:

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-04 Thread Greg Nancarrow
On Thu, Mar 4, 2021 at 11:05 PM Amit Kapila wrote: > > On Thu, Mar 4, 2021 at 5:24 PM Greg Nancarrow wrote: > > > > On Thu, Mar 4, 2021 at 10:07 PM Amit Kapila wrote: > > > > > > > >Also, in > > > standard_planner, we should add these partitionOids only for > > > parallel-mode. > > > > > > > It

Re: PROXY protocol support

2021-03-04 Thread Jacob Champion
On Thu, 2021-03-04 at 21:45 +0100, Magnus Hagander wrote: > On Thu, Mar 4, 2021 at 9:07 PM Jacob Champion wrote: > > Idle thought I had while setting up a local test rig: Are there any > > compelling cases for allowing PROXY packets to arrive over Unix > > sockets? (By which I mean, the proxy is

Re: Replace buffer I/O locks with condition variables (reviving an old patch)

2021-03-04 Thread Thomas Munro
On Fri, Feb 26, 2021 at 7:08 PM Thomas Munro wrote: > Back in 2016, Robert Haas proposed to replace I/O locks with condition > variables[1]. Condition variables went in and have found lots of > uses, but this patch to replace a bunch of LWLocks and some busy > looping did not. Since then, it

Re: PROXY protocol support

2021-03-04 Thread Tatsuo Ishii
>> On Thu, 2021-03-04 at 10:42 +0900, Tatsuo Ishii wrote: >> > Is there any formal specification for the "a protocol common and very >> > light weight in proxies"? >> >> See >> >> https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt > > Yeah, it's currently in one of the comments, but

Re: Wired if-statement in gen_partprune_steps_internal

2021-03-04 Thread Ryan Lambert
On Wed, Mar 3, 2021 at 11:03 PM Amit Langote wrote: > Sorry, this seems to have totally slipped my mind. > > Attached is the patch I had promised. > > Also, I have updated the title of the CF entry to "Some cosmetic > improvements of partition pruning code", which I think is more > appropriate.

Re: libpq debug log

2021-03-04 Thread Tom Lane
"tsunakawa.ta...@fujitsu.com" writes: > From: Kyotaro Horiguchi >> Using (inCursor - inStart) as logCursor doesn't work correctly if tracing >> state >> desyncs. Once desync happens inStart can be moved at the timing that the >> tracing code doesn't expect. This requires (as I mentioned

Re: Disallow SSL compression?

2021-03-04 Thread Daniel Gustafsson
> On 4 Mar 2021, at 11:59, Michael Paquier wrote: > > On Wed, Mar 03, 2021 at 03:14:01PM +0100, Peter Eisentraut wrote: >> Per your other thread, you should also remove the environment variable. >> >> In postgres_fdw, I think commenting it out is not the right change. The >> other commented

Re: SPI return

2021-03-04 Thread Tom Lane
Patrick Handja writes: > I am not able to get the array returned by get_tuples function, and I am > thinking it's SPI_finish(). When I tried to print my array tuples itens > after SPI_finish(), It is not working. Indeed, SPI_finish() frees everything that was allocated by SPI functions,

Re: Disallow SSL compression?

2021-03-04 Thread Daniel Gustafsson
> On 3 Mar 2021, at 15:14, Peter Eisentraut > wrote: > > On 03.03.21 11:31, Daniel Gustafsson wrote: >>> On 26 Feb 2021, at 20:34, Daniel Gustafsson wrote: >>> Attached is a v2 which retains the sslcompression parameter for backwards >>> compatibility. >> And now a v3 which fixes an oversight

SPI return

2021-03-04 Thread Patrick Handja
Greetings, I have spent a couple of days working on separated lib in PostgreSql, and I am facing some issues with the return of data using SPI (Server Programming Interface). I have this simple get_tuples function used in the main function foobar. My example is very simple too (Using a previously

Re: pg_amcheck contrib application

2021-03-04 Thread Mark Dilger
> On Mar 4, 2021, at 2:04 PM, Peter Geoghegan wrote: > > On Thu, Mar 4, 2021 at 7:29 AM Robert Haas wrote: >> I think this whole approach is pretty suspect because the number of >> blocks in the relation can increase (by relation extension) or >> decrease (by VACUUM or TRUNCATE) between the

Re: PROXY protocol support

2021-03-04 Thread Magnus Hagander
On Thu, Mar 4, 2021 at 10:01 PM Jan Wieck wrote: > > On 3/4/21 3:40 PM, Magnus Hagander wrote: > > On Thu, Mar 4, 2021 at 9:29 PM Jan Wieck wrote: > >> This looks like it would only need a few extra protocol messages to be > >> understood by the backend. It might be possible to implement that

Re: Fix DROP TABLESPACE on Windows with ProcSignalBarrier?

2021-03-04 Thread Thomas Munro
On Thu, Mar 4, 2021 at 11:54 AM Thomas Munro wrote: > > I've been wondering what obscure hazards these "tombstone" (for want > > of a better word) files guard against, besides the one described in > > the comments for mdunlink(). I've been thinking about various > > schemes that can be

Re: pg_amcheck contrib application

2021-03-04 Thread Peter Geoghegan
On Thu, Mar 4, 2021 at 7:29 AM Robert Haas wrote: > I think this whole approach is pretty suspect because the number of > blocks in the relation can increase (by relation extension) or > decrease (by VACUUM or TRUNCATE) between the time when we query for > the list of target relations and the

Make relfile tombstone files conditional on WAL level

2021-03-04 Thread Thomas Munro
Hi, I'm starting a new thread for this patch that originated as a side-discussion in [1], to give it its own CF entry in the next cycle. This is a WIP with an open question to research: what could actually break if we did this? [1]

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-04 Thread Alvaro Herrera
On 2021-Mar-04, Alvaro Herrera wrote: > I don't know where do __WSAFDIsSet and __imp_select come from or what to > do about them. Let's see if adding pgport and pgcommon fixes things. Indeed all those other problems were fixed and these remain. New failure is:

Re: Removing support for COPY FROM STDIN in protocol version 2

2021-03-04 Thread Tom Lane
Heikki Linnakangas writes: > Joseph, any chance we could see a backtrace or some other details from > those crashes? +1 > 'drongo' just reported linker errors: > postgres.def : error LNK2001: unresolved external symbol > GetOldFunctionMessage >

Re: Add .log file ending to tap test log files

2021-03-04 Thread Alvaro Herrera
On 2021-Mar-04, Andres Freund wrote: > Right now it's harder than necessary to capture the log output from tap > tests because the the regression tests files don't end with a common > file ending with other types of logs. They're > # Open the test log file, whose name depends on the test

Re: [PATCH] postgres-fdw: column option to override foreign types

2021-03-04 Thread Tom Lane
"Dian M Fay" writes: > On Thu Mar 4, 2021 at 9:28 AM EST, Georgios Kokolatos wrote: >> I am afraid I will have to :-1: this patch. > I see room for interpretation in the design here, although I have > admittedly not been looking at it for very long. `CREATE/ALTER FOREIGN > TABLE` take the user

Re: pg_amcheck contrib application

2021-03-04 Thread Robert Haas
On Thu, Mar 4, 2021 at 12:27 PM Robert Haas wrote: > More in a bit, need to grab some lunch. Moving on to the tests, in 003_check.pl, I think it would be slightly better if relation_toast were to select ct.oid::regclass and then just have the caller use that value directly. We'd certainly want

Re: Removing support for COPY FROM STDIN in protocol version 2

2021-03-04 Thread Andrew Dunstan
On 3/4/21 3:55 PM, Heikki Linnakangas wrote: > > > > > 'drongo' just reported linker errors: > > postgres.def : error LNK2001: unresolved external symbol > GetOldFunctionMessage > [c:\\prog\\bf\\root\\HEAD\\pgsql.build\\postgres.vcxproj] > postgres.def : error LNK2001: unresolved external symbol

Re: CI/windows docker vs "am a service" autodetection on windows

2021-03-04 Thread Alvaro Herrera
On 2021-Mar-04, Andres Freund wrote: > > > There does seem to be isatty(), so we could improve the case of > > > pg_ctl/postgres run interactively without breaking a sweat. And there is > > > fstat() too, so if stderr in a service is something distinguishable... > > > > We seem to have used that

Re: PROXY protocol support

2021-03-04 Thread Jan Wieck
On 3/4/21 3:40 PM, Magnus Hagander wrote: On Thu, Mar 4, 2021 at 9:29 PM Jan Wieck wrote: This looks like it would only need a few extra protocol messages to be understood by the backend. It might be possible to implement that with the loadable wire protocol extensions proposed here:

Re: Huge memory consumption on partitioned table with FKs

2021-03-04 Thread Tom Lane
Amit Langote writes: > Updated patch attached. This claim seems false on its face: > All child constraints of a given foreign key constraint can use the > same RI query and the resulting plan, that is, no need to create as > many copies of the query and the plan as there are partitions, as >

Re: Removing support for COPY FROM STDIN in protocol version 2

2021-03-04 Thread Heikki Linnakangas
On 04/03/2021 22:04, Tom Lane wrote: Heikki Linnakangas writes: I concur that 0001 attached is committable. I have not looked at your 0002, though. Removed the entry from nls.mk, and pushed 0001. Thanks! It seems that buildfarm member walleye doesn't like this. Since nothing else is

Re: Extensibility of the PostgreSQL wire protocol

2021-03-04 Thread Jan Wieck
On 3/3/21 2:43 PM, Peter Eisentraut wrote: I think, the way the abstractions are chosen in this patch, it is still very much tied to how the libpq protocol works. For example, there is a cancel key and a ready-for-query message. Some of the details of the simple and the extended query are

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-04 Thread Alvaro Herrera
On 2021-Mar-04, Alvaro Herrera wrote: > v31. Got this: libpq_pipeline.obj : error LNK2019: unresolved external symbol __WSAFDIsSet referenced in function test_pipelined_insert [C:\projects\postgresql\libpq_pipeline.vcxproj] 5019libpq_pipeline.obj : error LNK2019: unresolved external symbol

Re: CI/windows docker vs "am a service" autodetection on windows

2021-03-04 Thread Andres Freund
Hi, On 2021-03-04 21:36:23 +0100, Magnus Hagander wrote: > > I think that's a good answer for pg_ctl - not so sure about postgres > > itself, at least once it's up and running. I don't know what lead to all > > of this autodetection stuff, but is there the possibility of blocking on > > whatever

Re: PROXY protocol support

2021-03-04 Thread Magnus Hagander
On Thu, Mar 4, 2021 at 8:45 PM Jacob Champion wrote: > > On Thu, 2021-03-04 at 10:42 +0900, Tatsuo Ishii wrote: > > Is there any formal specification for the "a protocol common and very > > light weight in proxies"? > > See > > https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

Re: PROXY protocol support

2021-03-04 Thread Magnus Hagander
On Thu, Mar 4, 2021 at 9:07 PM Jacob Champion wrote: > > On Wed, 2021-03-03 at 10:39 +0100, Magnus Hagander wrote: > > On Wed, Mar 3, 2021 at 10:00 AM Magnus Hagander wrote: > > > Another option would of course be to listen on a separate port for it, > > > which seems to be the "haproxy way".

Re: PROXY protocol support

2021-03-04 Thread Magnus Hagander
On Thu, Mar 4, 2021 at 9:29 PM Jan Wieck wrote: > > On 3/4/21 2:45 PM, Jacob Champion wrote: > > On Thu, 2021-03-04 at 10:42 +0900, Tatsuo Ishii wrote: > >> Is there any formal specification for the "a protocol common and very > >> light weight in proxies"? > > > > See > > > >

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-03-04 Thread Justin Pryzby
I think this change to the regression tests is suspicous: -CONTEXT: remote SQL command: INSERT INTO public.loc2(f1, f2) VALUES ($1, $2) -COPY rem2, line 1: "-1 xyzzy" +CONTEXT: COPY loc2, line 1: "-1 xyzzy" +remote SQL command: COPY public.loc2(f1, f2) FROM STDIN +COPY rem2, line 2 I

Re: CI/windows docker vs "am a service" autodetection on windows

2021-03-04 Thread Magnus Hagander
On Thu, Mar 4, 2021 at 9:30 PM Andres Freund wrote: > > Hi, > > On 2021-03-04 21:08:30 +0100, Magnus Hagander wrote: > > The problem with doing it at register time is that everybody who > > builds an installer for PostgreSQL will then have to do it in their > > own registration (I'm pretty sure

Re: CI/windows docker vs "am a service" autodetection on windows

2021-03-04 Thread Andres Freund
Hi, On 2021-03-04 21:08:30 +0100, Magnus Hagander wrote: > The problem with doing it at register time is that everybody who > builds an installer for PostgreSQL will then have to do it in their > own registration (I'm pretty sure most of them don't use pg_ctl > register). Well, hm, maybe they

Re: PROXY protocol support

2021-03-04 Thread Jan Wieck
On 3/4/21 2:45 PM, Jacob Champion wrote: On Thu, 2021-03-04 at 10:42 +0900, Tatsuo Ishii wrote: Is there any formal specification for the "a protocol common and very light weight in proxies"? See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt which is maintained by HAProxy

Re: CI/windows docker vs "am a service" autodetection on windows

2021-03-04 Thread Magnus Hagander
On Thu, Mar 4, 2021 at 8:33 PM Andrew Dunstan wrote: > > > On 3/4/21 2:08 PM, Andres Freund wrote: > > [...] pgwin32_is_service() doesn't actually reliably detect if running as > > a service - it's a heuristic that also triggers when running postgres > > within a windows docker container

Re: PROXY protocol support

2021-03-04 Thread Jacob Champion
On Wed, 2021-03-03 at 10:39 +0100, Magnus Hagander wrote: > On Wed, Mar 3, 2021 at 10:00 AM Magnus Hagander wrote: > > Another option would of course be to listen on a separate port for it, > > which seems to be the "haproxy way". That would be slightly more code > > (we'd still want to keep the

Re: Removing support for COPY FROM STDIN in protocol version 2

2021-03-04 Thread Tom Lane
Heikki Linnakangas writes: >> I concur that 0001 attached is committable. I have not looked at >> your 0002, though. > Removed the entry from nls.mk, and pushed 0001. Thanks! It seems that buildfarm member walleye doesn't like this. Since nothing else is complaining, I confess bafflement as to

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-04 Thread Alvaro Herrera
On 2021-Mar-04, Alvaro Herrera wrote: > I think the problem is that the project is called pipeline and not > test_libpq, so there's no match in the name. I'm going to rename the > whole thing to src/test/modules/libpq_pipeline/ and see if the msvc > tooling likes that better. v31. -- Álvaro

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-04 Thread Alvaro Herrera
On 2021-Mar-04, Alvaro Herrera wrote: > v30 contains changes to hopefully make it build on MSVC. Hm, that didn't work -- appveyor still says: Project "C:\projects\postgresql\pgsql.sln" (1) is building "C:\projects\postgresql\pipeline.vcxproj" (75) on node 1 (default targets). PrepareForBuild:

Re: PROXY protocol support

2021-03-04 Thread Jacob Champion
On Thu, 2021-03-04 at 10:42 +0900, Tatsuo Ishii wrote: > Is there any formal specification for the "a protocol common and very > light weight in proxies"? See https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt which is maintained by HAProxy Technologies. --Jacob

  1   2   >