Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-28 Thread Amit Kapila
On Sun, Sep 29, 2019 at 12:39 AM Tomas Vondra wrote: > > On Sat, Sep 28, 2019 at 01:36:46PM +0530, Amit Kapila wrote: > >On Fri, Sep 27, 2019 at 4:55 PM Tomas Vondra > > wrote: > > I do think having a separate GUC is a must, irrespectedly of what other > GUC (if any) is used as a default. You're

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-28 Thread Dilip Kumar
On Thu, Sep 26, 2019 at 11:38 PM Tomas Vondra wrote: > > No, that's a good question, and I'm not sure what the answer is at the > moment. My understanding was that the infrastructure in the 2PC patch is > enough even for subtransactions, but I might be wrong. I need to think > about that for a

Re: [DOC] Document concurrent index builds waiting on each other

2019-09-28 Thread James Coleman
On Sat, Sep 28, 2019 at 9:56 PM Bruce Momjian wrote: > > On Sat, Sep 28, 2019 at 09:54:48PM -0400, James Coleman wrote: > > On Sat, Sep 28, 2019 at 9:22 PM Alvaro Herrera > > wrote: > > > > > > On 2019-Sep-28, Bruce Momjian wrote: > > > > > > > The CREATE INDEX docs already say: > > > > > > > >

Re: [DOC] Document concurrent index builds waiting on each other

2019-09-28 Thread Bruce Momjian
On Sat, Sep 28, 2019 at 09:54:48PM -0400, James Coleman wrote: > On Sat, Sep 28, 2019 at 9:22 PM Alvaro Herrera > wrote: > > > > On 2019-Sep-28, Bruce Momjian wrote: > > > > > The CREATE INDEX docs already say: > > > > > > In a concurrent index build, the index is actually entered into > > >

Re: [DOC] Document concurrent index builds waiting on each other

2019-09-28 Thread James Coleman
On Sat, Sep 28, 2019 at 9:22 PM Alvaro Herrera wrote: > > On 2019-Sep-28, Bruce Momjian wrote: > > > The CREATE INDEX docs already say: > > > > In a concurrent index build, the index is actually entered into > > the system catalogs in one transaction, then two table scans occur in > >

Re: PostgreSQL12 and older versions of OpenSSL

2019-09-28 Thread Michael Paquier
On Sat, Sep 28, 2019 at 10:52:18PM +0200, Peter Eisentraut wrote: > committed with that Thanks, LGTM. -- Michael signature.asc Description: PGP signature

Re: Wrong results using initcap() with non normalized string

2019-09-28 Thread Alvaro Herrera
On 2019-Sep-22, Juan José Santamaría Flecha wrote: > The attached patch addresses the comment about assuming UTF8. The UTF8 bits looks reasonable to me. I guess the other part of that question is whether we support any other multibyte encoding that supports combining characters. Maybe for

Re: [DOC] Document concurrent index builds waiting on each other

2019-09-28 Thread Alvaro Herrera
On 2019-Sep-28, Bruce Momjian wrote: > The CREATE INDEX docs already say: > > In a concurrent index build, the index is actually entered into > the system catalogs in one transaction, then two table scans occur in > two more transactions. Before each table scan, the index build must

Re: Consider low startup cost in add_partial_path

2019-09-28 Thread James Coleman
On Saturday, September 28, 2019, Tomas Vondra wrote: > On Sat, Sep 28, 2019 at 12:16:05AM -0400, Robert Haas wrote: > >> On Fri, Sep 27, 2019 at 2:24 PM James Coleman wrote: >> >>> Over in the incremental sort patch discussion we found [1] a case >>> where a higher cost plan ends up being

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-09-28 Thread Tomas Vondra
On Fri, Sep 27, 2019 at 01:50:30PM -0400, James Coleman wrote: On Mon, Sep 9, 2019 at 5:55 PM Tomas Vondra wrote: The "patched" column means all developer GUCs disabled, so it's expected to produce the same plan as master (and it is). And then there's one column for each developer GUC. If the

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-09-28 Thread Tomas Vondra
On Fri, Sep 27, 2019 at 08:31:30PM -0400, James Coleman wrote: On Fri, Sep 13, 2019 at 10:51 AM Tomas Vondra wrote: On Thu, Sep 12, 2019 at 12:49:29PM -0300, Alvaro Herrera wrote: >On 2019-Jul-30, Tomas Vondra wrote: >> I've decided to do a couple of experiments, trying to make my mind about

Re: Consider low startup cost in add_partial_path

2019-09-28 Thread Tomas Vondra
On Sat, Sep 28, 2019 at 12:16:05AM -0400, Robert Haas wrote: On Fri, Sep 27, 2019 at 2:24 PM James Coleman wrote: Over in the incremental sort patch discussion we found [1] a case where a higher cost plan ends up being chosen because a low startup cost partial path is ignored in favor of a

Re: Memory Accounting

2019-09-28 Thread Tomas Vondra
On Sun, Sep 29, 2019 at 12:12:49AM +0200, Tomas Vondra wrote: On Thu, Sep 26, 2019 at 01:36:46PM -0700, Jeff Davis wrote: On Thu, 2019-09-26 at 21:22 +0200, Tomas Vondra wrote: It's worth mentioning that those bechmarks (I'm assuming we're talking about the numbers Rober shared in [1]) were

Re: Memory Accounting

2019-09-28 Thread Tomas Vondra
On Thu, Sep 26, 2019 at 01:36:46PM -0700, Jeff Davis wrote: On Thu, 2019-09-26 at 21:22 +0200, Tomas Vondra wrote: It's worth mentioning that those bechmarks (I'm assuming we're talking about the numbers Rober shared in [1]) were done on patches that used the eager accounting approach (i.e.

Re: Standby accepts recovery_target_timeline setting?

2019-09-28 Thread Peter Eisentraut
On 2019-09-28 19:45, Tom Lane wrote: > Maybe I'm misunderstanding, but I think that rather than adding error > checks that were not there before, the right path to fixing this is > to cause these settings to be ignored if we're doing crash recovery. That makes sense to me. Something like this

Re: PostgreSQL12 and older versions of OpenSSL

2019-09-28 Thread Peter Eisentraut
On 2019-09-27 16:20, Michael Paquier wrote: > On Fri, Sep 27, 2019 at 03:50:57PM +0200, Peter Eisentraut wrote: >> On 2019-09-27 03:51, Michael Paquier wrote: >>> Your patch does not issue a ereport(LOG/FATAL) in the event of a >>> failure with SSL_CTX_set_max_proto_version(), which is something

Re: Usage of the system truststore for SSL certificate validation

2019-09-28 Thread Bruce Momjian
On Thu, Sep 19, 2019 at 12:26:27PM -0400, Isaac Morland wrote: > If we're going to open this up, can we add an option to say "this key is > allowed to log in to this account", SSH style? > > I like the idea of using keys rather than .pgpass, but I like the ~/.ssh/ > authorized_keys model and

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-28 Thread Tomas Vondra
On Sat, Sep 28, 2019 at 01:36:46PM +0530, Amit Kapila wrote: On Fri, Sep 27, 2019 at 4:55 PM Tomas Vondra wrote: On Fri, Sep 27, 2019 at 02:33:32PM +0530, Amit Kapila wrote: >On Tue, Jan 9, 2018 at 7:55 AM Peter Eisentraut > wrote: >> >> On 1/3/18 14:53, Tomas Vondra wrote: >> >> I don't see

Re: Standby accepts recovery_target_timeline setting?

2019-09-28 Thread David Steele
On 9/28/19 1:26 PM, Fujii Masao wrote: > On Sun, Sep 29, 2019 at 12:51 AM David Steele wrote: > > Yeah, more checks would be necessary. IMO easy fix is to forbid not only > recovery target parameters but also any recovery parameters (specified > in recovery.conf in previous versions) in crash

Re: Standby accepts recovery_target_timeline setting?

2019-09-28 Thread Tom Lane
Fujii Masao writes: >> Agreed. Seems like that could be added to the patch above easily >> enough. More checks would be needed to prevent the behaviors I've been >> seeing in the other thread, but it should be possible to more or less >> mimic the old behavior with sufficient checks. > Yeah,

Re: Unstable select_parallel regression output in 12rc1

2019-09-28 Thread Tom Lane
Christoph Berg writes: > Re: Tom Lane 2019-09-26 <12685.1569510...@sss.pgh.pa.us> >> We haven't seen it in quite some time in HEAD, though I fear that's >> just due to bad luck or change of timing of unrelated tests. > The v13 package builds that are running every 6h here haven't seen a >

Re: max_parallel_workers question

2019-09-28 Thread Jeff Davis
On Sat, 2019-09-28 at 00:10 -0400, Robert Haas wrote: > I intended it to mean "the entire cluster." Basically, how many > workers out of max_worker_processes are you willing to use for > parallel query, as opposed to other things. I agree that PGC_USERSET > doesn't make any sense. In that case,

Re: Instability of partition_prune regression test results

2019-09-28 Thread Tom Lane
Amit Langote writes: > On Sat, Sep 28, 2019 at 12:59 AM Tom Lane wrote: >> Amit Langote writes: >>> Isn't the point of using ANALYZE here to show that the exec-param >>> based run-time pruning is working (those "never executed" strings)? >> Hm. Well, if you want to see those, we could do it

Re: Standby accepts recovery_target_timeline setting?

2019-09-28 Thread Fujii Masao
On Sun, Sep 29, 2019 at 12:51 AM David Steele wrote: > > On 9/28/19 10:54 AM, Fujii Masao wrote: > > On Sat, Sep 28, 2019 at 2:01 AM David Steele wrote: > >> On 9/27/19 11:58 AM, Fujii Masao wrote: > >>> > >>> Yes, recovery target settings are used even when neither backup_label > >>> nor

Re: Document recovery_target_action behavior?

2019-09-28 Thread Jonathan S. Katz
On 9/28/19 12:00 PM, David Steele wrote: > On 9/28/19 11:14 AM, Fujii Masao wrote: >> On Sat, Sep 28, 2019 at 2:52 AM David Steele wrote: >> >>> The question for the old versions: is this something that should be >>> fixed in the code or in the documentation? >>> >>> My vote is to make this

Re: [DOC] Document concurrent index builds waiting on each other

2019-09-28 Thread Bruce Momjian
On Wed, Sep 18, 2019 at 01:51:00PM -0400, James Coleman wrote: > In my experience it's not immediately obvious (even after reading the > documentation) the implications of how concurrent index builds manage > transactions with respect to multiple concurrent index builds in > flight at the same

Re: Document recovery_target_action behavior?

2019-09-28 Thread David Steele
On 9/28/19 11:14 AM, Fujii Masao wrote: > On Sat, Sep 28, 2019 at 2:52 AM David Steele wrote: > >> The question for the old versions: is this something that should be >> fixed in the code or in the documentation? >> >> My vote is to make this explicit in the documentation, since changing >> the

Re: Standby accepts recovery_target_timeline setting?

2019-09-28 Thread David Steele
On 9/28/19 10:54 AM, Fujii Masao wrote: > On Sat, Sep 28, 2019 at 2:01 AM David Steele wrote: >> On 9/27/19 11:58 AM, Fujii Masao wrote: >>> >>> Yes, recovery target settings are used even when neither backup_label >>> nor recovery.signal exist, i.e., just a crash recovery, in v12. This is >>>

default partitions can be partitioned and have default partitions?

2019-09-28 Thread Justin Pryzby
postgres=# CREATE TABLE t(i int)PARTITION BY RANGE(i); CREATE TABLE postgres=# CREATE TABLE t0 PARTITION OF t DEFAULT PARTITION BY RANGE(i); CREATE TABLE postgres=# CREATE TABLE t00 PARTITION OF t0 DEFAULT; -- oh yes CREATE TABLE ... Not sure how it could be useful to partition default into

Re: Document recovery_target_action behavior?

2019-09-28 Thread Fujii Masao
On Sat, Sep 28, 2019 at 2:52 AM David Steele wrote: > > Hackers, > > In versions < PG12 recovery_target_action has a behavior that appears to > be a bug, or is at least undocumented. If hot_standby = off and > recovery_target_action is not specified then the cluster will promote > when the

Re: Standby accepts recovery_target_timeline setting?

2019-09-28 Thread Fujii Masao
On Sat, Sep 28, 2019 at 2:01 AM David Steele wrote: > > On 9/27/19 11:58 AM, Fujii Masao wrote: > > On Sat, Sep 28, 2019 at 12:14 AM David Steele wrote: > >> > >> I think, at the very least, the fact that targeted recovery proceeds in > >> the absence of recovery.signal represents a bug. > > > >

Re: Hooks for session start and end, take two

2019-09-28 Thread Fabrízio de Royes Mello
On Fri, Sep 27, 2019 at 4:26 PM legrand legrand wrote: > > OK I confirm: > - "client backend" appears at session start and end hook, > - "autovacuum worker" and "pg_background" only appears at session end hook > (backend_start can be retreived from pg_stat_activity), > - "parallel workers" are

Re: Attempt to consolidate reading of XLOG page

2019-09-28 Thread Antonin Houska
Antonin Houska wrote: > Alvaro Herrera wrote: > > > BTW that tli_p business to the openSegment callback is horribly > > inconsistent. Some callers accept a NULL tli_p, others will outright > > crash, even though the API docs say that the callback must determine the > > timeline. This is made

Re: Modest proposal for making bpchar less inconsistent

2019-09-28 Thread Bruce Momjian
On Fri, Sep 13, 2019 at 09:50:10PM +0200, Pavel Stehule wrote: > > > Dne pá 13. 9. 2019 16:43 uživatel Tom Lane napsal: > > It struck me that the real reason that we keep getting gripes about > the weird behavior of CHAR(n) is that these functions (and, hence, > their corresponding

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-28 Thread Amit Kapila
On Thu, Sep 26, 2019 at 11:37 PM Tomas Vondra wrote: > > Hi, > > Attached is an updated patch series, rebased on current master. It does > fix one memory accounting bug in ReorderBufferToastReplace (the code was > not properly updating the amount of memory). > Few comments on 0001 1. I am

Re: pglz performance

2019-09-28 Thread Andrey Borodin
Oleg, Peter, thanks for looking into this! I hope to benchmark decompression on Silesian corpus soon. PFA v2 with better comments. > 27 сент. 2019 г., в 14:41, Peter Eisentraut > написал(а): > > After reviewing this thread and more testing, I think >

Re: Attempt to consolidate reading of XLOG page

2019-09-28 Thread Antonin Houska
Tom Lane wrote: > Alvaro Herrera writes: > > On 2019-Sep-27, Antonin Houska wrote: > >>> You placed the errinfo in XLogRead's stack rather than its callers' ... > >>> I don't think that works, because as soon as XLogRead returns that > >>> memory is no longer guaranteed to exist. > > >> I was

Re: Attempt to consolidate reading of XLOG page

2019-09-28 Thread Antonin Houska
Alvaro Herrera wrote: > BTW that tli_p business to the openSegment callback is horribly > inconsistent. Some callers accept a NULL tli_p, others will outright > crash, even though the API docs say that the callback must determine the > timeline. This is made more complicated by us having the

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-09-28 Thread Amit Kapila
On Fri, Sep 27, 2019 at 4:55 PM Tomas Vondra wrote: > > On Fri, Sep 27, 2019 at 02:33:32PM +0530, Amit Kapila wrote: > >On Tue, Jan 9, 2018 at 7:55 AM Peter Eisentraut > > wrote: > >> > >> On 1/3/18 14:53, Tomas Vondra wrote: > >> >> I don't see the need to tie this setting to

Re: Instability of partition_prune regression test results

2019-09-28 Thread Amit Langote
On Sat, Sep 28, 2019 at 12:59 AM Tom Lane wrote: > Amit Langote writes: > > On Fri, Sep 27, 2019 at 7:25 AM Tom Lane wrote: > >> I experimented with adjusting explain_parallel_append() to filter > >> more fields, but soon realized that we'd have to filter out basically > >> everything that

Re: Cleanup code related to OpenSSL <= 0.9.6 in fe/be-secure-openssl.c

2019-09-28 Thread Michael Paquier
On Fri, Sep 27, 2019 at 03:46:09PM +0200, Peter Eisentraut wrote: > Yes, it seems OK to clean this up in master. Thanks, applied on HEAD. -- Michael signature.asc Description: PGP signature

Re: pgbench - allow to create partitioned tables

2019-09-28 Thread Fabien COELHO
Hello Amit, I think we might also need to use pg_get_partkeydef along with pg_partition_tree to fetch the partition method information. However, I think to find reloid of pgbench_accounts in the current search path, we might need to use some part of query constructed by Fabien. Fabien, what

Re: Change atoi to strtol in same place

2019-09-28 Thread Michael Paquier
On Fri, Sep 27, 2019 at 09:35:53PM -0500, Joe Nelson wrote: > Note that it requires functions from str2int-10.patch, and will not > compile when applied to master by itself. I didn't want to duplicate > functionality from that other uncommitted patch in mine. If we have a dependency between both

Re: Don't codegen deform code for virtual tuples in expr eval for scan fetch

2019-09-28 Thread Soumyadeep Chakraborty
Hey, I completely agree, that was an important consideration. I had some purely cosmetic suggestions: 1. Rename ExecComputeSlotInfo to eliminate the need for the asserts. 2. Extract return value to a bool variable for slightly better readability. 3. Taking the opportunity to use TTS_IS_VIRTUAL.

Re: contrib/bloom Valgrind error

2019-09-28 Thread Michael Paquier
On Fri, Sep 27, 2019 at 09:02:34PM -0700, Peter Geoghegan wrote: > My Valgrind test script reports the following error, triggered from > within contrib/bloom's regression test suite on master as of right > now: > > I suspect that the recent commit 69f94108 is involved here, but I > haven't