Re: Postgres is not able to handle more than 4k tables!?

2020-07-10 Thread Konstantin Knizhnik
On 09.07.2020 22:16, Grigory Smolkin wrote: On 7/8/20 11:41 PM, Konstantin Knizhnik wrote: So looks like NUM_LOCK_PARTITIONS and MAXNUMMESSAGES  constants have to be replaced with GUCs. To avoid division, we can specify log2 of this values, so shift can be used instead. And MAX_SIMUL_LWLOC

Re: Postgres is not able to handle more than 4k tables!?

2020-07-10 Thread Laurenz Albe
On Thu, 2020-07-09 at 12:47 -0400, Stephen Frost wrote: > I realize this is likely to go over like a lead balloon, but the churn > in pg_class from updating reltuples/relpages has never seemed all that > great to me when just about everything else is so rarely changed, and > only through some user

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-07-10 Thread Kasahara Tatsuhito
Hi, On Wed, Jul 8, 2020 at 9:40 PM Bharath Rupireddy wrote: > One way, we could solve the above problem is that, upon firing the new > foreign query from local backend using the cached connection, > (assuming the remote backend that was cached in the local backed got > killed for some reasons), i

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Jeff Davis
On Thu, 2020-07-09 at 19:18 -0500, Justin Pryzby wrote: > Maybe pretend that Jeff implemented something called CashAgg, which > does > everything HashAgg does but implemented from scratch. Users would be > able to > tune it or disable it, and we could talk about removing HashAgg for > the next 3 >

Re: Creating a function for exposing memory usage of backend process

2020-07-10 Thread torikoshia
On 2020-07-08 22:12, Fujii Masao wrote: Thanks for updating the patch! It basically looks good to me. + + backend memory contexts + Do we need this indexterm? Thanks! it's not necessary. I remove this indexterm. +{ oid => '2282', descr => 'statistics: information about all memory co

Re: Creating a function for exposing memory usage of backend process

2020-07-10 Thread torikoshia
On 2020-07-09 02:03, Andres Freund wrote: Hi, I think this is an incredibly useful feature. Thanks for your kind comments and suggestion! On 2020-07-07 22:02:10 +0900, torikoshia wrote: > There can be multiple memory contexts with the same name. So I'm afraid > that it's difficult to ident

GSSENC'ed connection stalls while reconnection attempts.

2020-07-10 Thread Kyotaro Horiguchi
Hello. If psql connected using GSSAPI auth and server restarted, reconnection sequence stalls and won't return. I found that psql(libpq) sends startup packet via gss encryption. conn->gssenc should be reset when encryption state is freed. The reason that psql doesn't notice the error is pqPacket

Re: WIP: BRIN multi-range indexes

2020-07-10 Thread Masahiko Sawada
On Fri, 3 Jul 2020 at 09:58, Tomas Vondra wrote: > > On Sun, Apr 05, 2020 at 08:01:50PM +0300, Alexander Korotkov wrote: > >On Sun, Apr 5, 2020 at 8:00 PM Tomas Vondra > > wrote: > ... > >> > > >> >Assuming we're not going to get 0001-0003 into v13, I'm not so > >> >inclined to rush on these three

Re: [Proposal] Global temporary tables

2020-07-10 Thread wenjing zeng
HI all I started using my personal email to respond to community issue. > 2020年7月7日 下午6:05,Pavel Stehule 写道: > > Hi > > GTT Merge the latest PGMaster and resolves conflicts. > > > > I tested it and it looks fine. I think it is very usable in current form, but > still there are some issu

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-10 Thread Amit Kapila
On Fri, Jul 10, 2020 at 7:19 AM Masahiko Sawada wrote: > > On Thu, 9 Jul 2020 at 16:09, Amit Kapila wrote: > > > > > > Fair enough. The attached patch reverts the commits related to these > > stats. Sawada-San, can you please once see if I have missed anything > > apart from catversion bump whi

Re: Resetting spilled txn statistics in pg_stat_replication

2020-07-10 Thread Amit Kapila
On Fri, Jul 10, 2020 at 7:23 AM Masahiko Sawada wrote: > > On Thu, 9 Jul 2020 at 12:11, Amit Kapila wrote: > > > > On Wed, Jul 8, 2020 at 1:14 PM Masahiko Sawada > > wrote: > > > > > > > > > I think that using oids has another benefit that we don't need to send > > > slot name to the stats colle

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-07-10 Thread Dilip Kumar
On Sat, Jul 4, 2020 at 11:35 AM Amit Kapila wrote: > > On Tue, Jun 30, 2020 at 5:20 PM Amit Kapila wrote: > > > > Let me know what you think about the above changes. > > > > I went ahead and made few changes in > 0005-Implement-streaming-mode-in-ReorderBuffer which are explained > below. I have

Re: [PATCH v2] Allow COPY "text" to output a header and add header matching mode to COPY FROM

2020-07-10 Thread Peter Eisentraut
On 2020-07-08 13:45, Rémi Lapeyre wrote: Hi, here's a new version of the patch that should apply cleanly. I'll monitor the status on http://cfbot.cputube.org/ It's hard to find an explanation what this patch actually does. I don't want to have to go through threads dating back 4 months to det

Re: posgres 12 bug (partitioned table)

2020-07-10 Thread Amit Langote
Reading my own words, I think I must fix an ambiguity: On Fri, Jul 10, 2020 at 3:23 PM Amit Langote wrote: > So even if an AM's table_tuple_insert() itself doesn't populate the > transaction info into the slot handed to it, maybe as an optimization, > it does not sound entirely unreasonable to ex

Re: TAP tests and symlinks on Windows

2020-07-10 Thread Andrew Dunstan
On 7/8/20 9:54 AM, Andrew Dunstan wrote: > > > > Then, with a little more sprinkling of perl2host the pg_basebackup tests > can be made to work on msys2. > > > I'm going to prepare patches along these lines. > > After much frustration and gnashing of teeth here's a patch that allows almost all

Re: WIP: BRIN multi-range indexes

2020-07-10 Thread Tomas Vondra
On Fri, Jul 10, 2020 at 06:01:58PM +0900, Masahiko Sawada wrote: On Fri, 3 Jul 2020 at 09:58, Tomas Vondra wrote: On Sun, Apr 05, 2020 at 08:01:50PM +0300, Alexander Korotkov wrote: >On Sun, Apr 5, 2020 at 8:00 PM Tomas Vondra > wrote: ... >> > >> >Assuming we're not going to get 0001-0003 int

Re: Auto-vectorization speeds up multiplication of large-precision numerics

2020-07-10 Thread Peter Eisentraut
On 2020-06-10 14:15, Amit Khandekar wrote: Well, how do we make sure we keep it that way? How do we prevent some random rearranging of the code or some random compiler change to break this again? I believe the compiler rearranges the code segments w.r.t. one another when those are independent o

Re: Default gucs for EXPLAIN

2020-07-10 Thread Peter Eisentraut
On 2020-05-23 11:14, Vik Fearing wrote: Here is a patch to provide default gucs for EXPLAIN options. I have two goals with this patch. The first is that I personally *always* want BUFFERS turned on, so this would allow me to do it without typing it every time. There was a lot of opposition to

Re: Auto-vectorization speeds up multiplication of large-precision numerics

2020-07-10 Thread Tom Lane
Peter Eisentraut writes: > We normally don't compile with -O3, so very few users would get the > benefit of this. Yeah. I don't think changing that baseline globally would be a wise move. > We have CFLAGS_VECTOR for the checksum code. I > suppose if we are making the numeric code vectorizabl

Re: pg_regress cleans up tablespace twice.

2020-07-10 Thread Tom Lane
Thomas Munro writes: > On Thu, Jun 18, 2020 at 1:42 PM Michael Paquier wrote: >> Thanks, applied this part to HEAD then after more testing. > Hmm, somehow this (well I guess it's this commit based on timing and > the area it touches, not sure exactly why) made cfbot's Windows build > fail, like

Re: pgbench - add pseudo-random permutation function

2020-07-10 Thread Fabien COELHO
Attached v19 is a rebase, per cfbot. Attached v20 fixes a doc xml typo, per cfbot again. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 9f3bb5fce6..d4a604c6fa 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -1033,7

RE: Implement UNLOGGED clause for COPY FROM

2020-07-10 Thread osumi.takami...@fujitsu.com
Hi David Johnston Thank you for your comment. Aside from that, though, how does this improve upon the existing capability to copy into an unlogged temporary table? [>] unlogged temporary table can’t be inherited over sessions first of all. And unlogged table needs to be recreated due to startup

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Stephen Frost
Greetings, * Justin Pryzby (pry...@telsasoft.com) wrote: > On Thu, Jul 09, 2020 at 06:58:40PM -0400, Stephen Frost wrote: > > * Peter Geoghegan (p...@bowt.ie) wrote: > > > On Thu, Jul 9, 2020 at 7:03 AM Stephen Frost wrote: > > > It makes more sense than simply ignoring what our users will see as

Re: Implement UNLOGGED clause for COPY FROM

2020-07-10 Thread Tom Lane
"osumi.takami...@fujitsu.com" writes: >> Aside from that, though, how does this improve upon the existing capability >> to copy into an unlogged temporary table? > [>] unlogged temporary table can’t be inherited over sessions first of all. Unlogged tables don't have to be temporary. > And unlo

Re: distribute_restrictinfo_to_rels if restrictinfo contains volatile functions

2020-07-10 Thread Tom Lane
Zhenghua Lyu writes: > The where clause is "pushed down to the x,y" because it only > references these two relations. Yeah. I agree that it's somewhat unprincipled, but changing it doesn't seem like a great idea. There are a lot of users out there who aren't terribly careful about mark

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Thu, Jul 9, 2020 at 5:08 PM Stephen Frost wrote: > > I didn't, and don't, think it particularly relevant to the discussion, > > but if you don't like the comparison to Sort then we could compare it to > > a HashJoin instead- the point is tha

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Stephen Frost
Greetings, * Jeff Davis (pg...@j-davis.com) wrote: > In principle, Stephen is right: the v12 behavior is a bug, lots of > people are unhappy about it, it causes real problems, and it would not > be acceptable if proposed today. Otherwise I wouldn't have spent the > time to fix it. > > Similarly,

Re: WIP: BRIN multi-range indexes

2020-07-10 Thread Sascha Kuhl
Tomas Vondra schrieb am Fr., 10. Juli 2020, 14:09: > On Fri, Jul 10, 2020 at 06:01:58PM +0900, Masahiko Sawada wrote: > >On Fri, 3 Jul 2020 at 09:58, Tomas Vondra > wrote: > >> > >> On Sun, Apr 05, 2020 at 08:01:50PM +0300, Alexander Korotkov wrote: > >> >On Sun, Apr 5, 2020 at 8:00 PM Tomas Von

libpq: Request Pipelining/Batching status ?

2020-07-10 Thread Matthieu Garrigues
Hi all, Do you know what is the status of Request Pipelining and/or Batching in libpq ? I could see that I'm not the first one to think about it, I see an item in the todolist: https://web.archive.org/web/20200125013930/https://wiki.postgresql.org/wiki/Todo And a thread here: https://www.postgre

Re: vs formatting in the docs

2020-07-10 Thread Peter Eisentraut
On 2020-06-21 18:57, Dagfinn Ilmari Mannsåker wrote: Attached are two patches: the first adds the missing tags, the second adds to all the SQL commands (specifically anything with 7). I have committed the first one. I have some concerns about the second one. If you look at the diff of the

Re: expose parallel leader in CSV and log_line_prefix

2020-07-10 Thread Julien Rouhaud
On Thu, Jul 09, 2020 at 09:20:23PM -0500, Justin Pryzby wrote: > On Fri, Jul 10, 2020 at 11:09:40AM +0900, Michael Paquier wrote: > > On Thu, Jul 09, 2020 at 01:53:39PM +0200, Julien Rouhaud wrote: > > > Sure! I've been quite busy with internal work duties recently but > > > I'll review this patch

Re: GSSENC'ed connection stalls while reconnection attempts.

2020-07-10 Thread Tom Lane
Kyotaro Horiguchi writes: > If psql connected using GSSAPI auth and server restarted, reconnection > sequence stalls and won't return. Yeah, reproduced here. (I wonder if there's any reasonable way to exercise this scenario in src/test/kerberos/.) > I found that psql(libpq) sends startup packet

Re: expose parallel leader in CSV and log_line_prefix

2020-07-10 Thread Justin Pryzby
On Fri, Jul 10, 2020 at 05:13:26PM +0200, Julien Rouhaud wrote: > There's a thinko in the padding handling. It should be dones whether MyProc > and/or lockGroupLeader is NULL or not, and only if padding was asked, like > it's > done for case 'd' for instance. > > Also, the '%k' escape sounds a b

Re: Stale external URL in doc?

2020-07-10 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Jul-10, Daniel Gustafsson wrote: >> Taking a look at other links to external resources, most links seemed to >> resolve still (but I didn't test all of them). I did find another one on the >> GEQO page which is now dead without the content available elsewhere, as

Re: expose parallel leader in CSV and log_line_prefix

2020-07-10 Thread Julien Rouhaud
On Fri, Jul 10, 2020 at 11:11:15AM -0500, Justin Pryzby wrote: > On Fri, Jul 10, 2020 at 05:13:26PM +0200, Julien Rouhaud wrote: > > There's a thinko in the padding handling. It should be dones whether MyProc > > and/or lockGroupLeader is NULL or not, and only if padding was asked, like > > it's

Re: expose parallel leader in CSV and log_line_prefix

2020-07-10 Thread Alvaro Herrera
On 2020-Mar-18, Justin Pryzby wrote: > On Sun, Mar 15, 2020 at 12:49:33PM +0100, Julien Rouhaud wrote: > template1=# SET log_temp_files=0; explain analyze SELECT a,COUNT(1) FROM t a > JOIN t b USING(a) GROUP BY 1; > 2020-03-15 21:20:47.288 CDT [55375537]LOG: statement: SET > log_temp_f

Re: expose parallel leader in CSV and log_line_prefix

2020-07-10 Thread Justin Pryzby
On Fri, Jul 10, 2020 at 12:45:29PM -0400, Alvaro Herrera wrote: > On 2020-Mar-18, Justin Pryzby wrote: > > > On Sun, Mar 15, 2020 at 12:49:33PM +0100, Julien Rouhaud wrote: > > > template1=# SET log_temp_files=0; explain analyze SELECT a,COUNT(1) FROM t > > a JOIN t b USING(a) GROUP BY 1; > > 20

RE: Index Skip Scan (new UniqueKeys)

2020-07-10 Thread Floris Van Nee
Hi Dmitry, Also took another look at the patch now, and found a case of incorrect data. It looks related to the new way of creating the paths, as I can't recall seeing this in earlier versions. create table t1 as select a,b,b%5 as c, random() as d from generate_series(1, 10) a, generate_series

Re: factorial function/phase out postfix operators?

2020-07-10 Thread Mark Dilger
> On Jun 30, 2020, at 2:47 PM, Mark Dilger wrote: > > > >> On May 19, 2020, at 4:47 PM, Tom Lane wrote: >> >> I wrote: >>> However, we do have to have a benefit to show those people whose >>> queries we break. Hence my insistence on having a working AS fix >>> (or some other benefit) befor

Re: expose parallel leader in CSV and log_line_prefix

2020-07-10 Thread Alvaro Herrera
On 2020-Jul-10, Justin Pryzby wrote: > On Fri, Jul 10, 2020 at 12:45:29PM -0400, Alvaro Herrera wrote: > > I think it's overly verbose; all non-parallel backends are going to get > > their own PID twice, and I'm not sure this is going to be great to > > parse. I think it would be more sensible th

Re: SQL-standard function body

2020-07-10 Thread Tom Lane
Thomas Munro writes: > On Wed, Jul 1, 2020 at 5:49 AM Peter Eisentraut > wrote: >> - More test coverage is needed. Surprisingly, there wasn't actually any >> test AFAICT that just creates and SQL function and runs it. Most of >> that code is tested incidentally, but there is very little or no >

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Peter Geoghegan
On Fri, Jul 10, 2020 at 7:17 AM Stephen Frost wrote: > > The hash_mem design (as it stands) would affect both hash join and > > hash aggregate. I believe that it makes most sense to have hash-based > > nodes under the control of a single GUC. I believe that this > > granularity will cause the leas

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Tom Lane
Peter Geoghegan writes: > On Fri, Jul 10, 2020 at 7:17 AM Stephen Frost wrote: >> Due to the fact that we're in beta and now is not the time to be >> redesigning this feature. > Did you read the discussion? Beta is when we fix problems that testing exposes in new features. Obviously, we'd rathe

Re: Report error position in partition bound check

2020-07-10 Thread Alexandra Wang
> On 2 July 2020, at 06:39, Daniel Gustafsson wrote: > > On 10 Apr 2020, at 23:50, Alexandra Wang wrote: > > > On Fri, Apr 10, 2020 at 8:37 AM Ashutosh Bapat < ashutosh.ba...@2ndquadrant.com > wrote: > > > for a multi-key value the ^ > > > points to the firs

Re: Binary support for pgoutput plugin

2020-07-10 Thread Tom Lane
Daniel Gustafsson writes: > Thanks for the update! Do note that my patch included a new file which is > missing from this patchset: > src/test/subscription/t/014_binary.pl > This is, IMO, the most interesting test of this feature so it would be good to > be included. It's a basic test and

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Jeff Davis
On Fri, 2020-07-10 at 13:46 -0400, Tom Lane wrote: > I looked over Peter's patch in [1], and it seems generally pretty > sane to me, though I concur with the idea that it'd be better to > define the GUC as a multiplier for work_mem. (For one thing, we > could > then easily limit it to be at least

Re: [Patch] Add missing libraries to Libs.private of libpq.pc

2020-07-10 Thread Peter Eisentraut
On 2020-04-08 11:38, Sandro Mani wrote: The following patch, which we added to build mingw-postgresql on Fedora, adds some missing libraries to Libs.private of libpq.pc, discovered when attempting to statically link with libpq: -lz: is required by -lcrypto I think the correct fix for that woul

Re: Remove Deprecated Exclusive Backup Mode

2020-07-10 Thread David Steele
On 7/2/20 7:12 AM, David Steele wrote: None of this really solves the problem of what happens when the user dumps the backup_label into the data directory. With traditional backup software that's pretty much going to be the only choice. Is telling them not to do it and washing our hands of it

Re: Created feature for to_date() conversion using patterns 'YYYY-WW', 'YYYY-WW-D', 'YYYY-MM-W' and 'YYYY-MM-W-D'

2020-07-10 Thread Tom Lane
I wrote: > Mark Lorenz writes: >> But, nevertheless, what about adding the function to accept the DAY, D >> (and maybe the Q) patterns for to_date() - in this case, of course, in >> the uncorrelated version? to_char() handles them properly. And, from my >> point of view, there is no reason why

Wait profiling

2020-07-10 Thread Daniel Wood
After nearly 5 years does something like the following yet exist? https://www.postgresql.org/message-id/559d4729.9080...@postgrespro.ru I feel that it would be useful to have the following two things. One PG enhancement and one standard extension. 1) An option to "explain" to produce a wait eve

Re: [PATCH] Btree BackwardScan race condition on Standby during VACUUM

2020-07-10 Thread Michail Nikolaev
Hello, Peter. Thanks for the update. Yes, it is the right decision. I have started to spot that bug only while working on a faster scan using hint bits on replicas [1], so it is unlikely to hit it in production at the moment. Thanks, Michail. [1]: https://www.postgresql.org/message-id/CANtu0oj

Re: Wait profiling

2020-07-10 Thread Alvaro Herrera
On 2020-Jul-10, Daniel Wood wrote: > After nearly 5 years does something like the following yet exist? > https://www.postgresql.org/message-id/559d4729.9080...@postgrespro.ru Yes, we have pg_stat_activity.wait_events which implement pretty much what Ildus describes there. > 1) An option to "expl

Re: "tuple concurrently updated" in pg_restore --jobs

2020-07-10 Thread Tom Lane
Justin Pryzby writes: > I hit this issue intermittently (roughly half the time) while working with a > patch David submitted, and finally found a recipe to reproduce it on an > unpatched v12 instance. > I was surprised to see pg_restore -j2 is restoring ACLs in pre-data in > parallel. It's not p

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Peter Geoghegan
On Fri, Jul 10, 2020 at 10:47 AM Tom Lane wrote: > I looked over Peter's patch in [1], and it seems generally pretty > sane to me, though I concur with the idea that it'd be better to > define the GUC as a multiplier for work_mem. (For one thing, we could > then easily limit it to be at least 1.0

Re: "tuple concurrently updated" in pg_restore --jobs

2020-07-10 Thread Justin Pryzby
On Fri, Jul 10, 2020 at 04:54:40PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I hit this issue intermittently (roughly half the time) while working with a > > patch David submitted, and finally found a recipe to reproduce it on an > > unpatched v12 instance. > > > I was surprised to see p

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Alvaro Herrera
On 2020-Jul-10, Peter Geoghegan wrote: > * The maximum allowable value is 100.0, to protect users from > accidentally setting hash_mem_multiplier to a value intended to work > like a work_mem-style KB value (you can't provide an absolute value > like that directly). This maximum is absurdly high.

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Peter Geoghegan
On Fri, Jul 10, 2020 at 11:34 AM Jeff Davis wrote: > On Fri, 2020-07-10 at 13:46 -0400, Tom Lane wrote: > > I looked over Peter's patch in [1], and it seems generally pretty > > sane to me, though I concur with the idea that it'd be better to > > define the GUC as a multiplier for work_mem. (For

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Peter Geoghegan
On Fri, Jul 10, 2020 at 2:10 PM Alvaro Herrera wrote: > I'm not sure about this bit; sounds a bit like what has been qualified > as "nannyism" elsewhere. Suppose I want to give a hash table 2GB of > memory for whatever reason. If my work_mem is default (4MB) then I > cannot possibly achieve that

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Jul-10, Peter Geoghegan wrote: >> * The maximum allowable value is 100.0, to protect users from >> accidentally setting hash_mem_multiplier to a value intended to work >> like a work_mem-style KB value (you can't provide an absolute value >> like that directly). Th

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Tom Lane
Peter Geoghegan writes: > It now seems likely that the hash_mem/hash_mem_multiplier proposal has > the support it needs to get into Postgres 13. Assuming that the > proposal doesn't lose momentum, then it's about time to return to the > original question you posed at the start of the thread: > Wh

Re: "tuple concurrently updated" in pg_restore --jobs

2020-07-10 Thread Tom Lane
Justin Pryzby writes: > On Fri, Jul 10, 2020 at 04:54:40PM -0400, Tom Lane wrote: >> This works about 99% of the time, in fact. It falls down in the --clean > Note that this fails for me (sometimes) even without --clean. Oh, I was thinking that REVOKE would only be issued in the --clean case, b

Re: Stale external URL in doc?

2020-07-10 Thread Daniel Gustafsson
> On 10 Jul 2020, at 18:28, Tom Lane wrote: > Alvaro Herrera writes: >> Um, the comp.ai.genetic FAQ can still be found, eg. >> http://www.faqs.org/faqs/ai-faq/genetic/part1/ > > So it is, although that also shows it hasn't been updated since 2001. Ah, I missed the alternative source. > I'll

Re: Stale external URL in doc?

2020-07-10 Thread Thomas Munro
On Fri, Jul 10, 2020 at 10:07 AM Daniel Gustafsson wrote: > (but I didn't test all of them) Cave-person shell script time: for url in ` git grep 'url="http' | sed 's/.*url="//;s/".*//' | sort | uniq ` do if ! curl --output /dev/null --silent --head --fail "$url" then echo "bad URL: $url"

Re: "tuple concurrently updated" in pg_restore --jobs

2020-07-10 Thread Justin Pryzby
On Fri, Jul 10, 2020 at 05:36:28PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Fri, Jul 10, 2020 at 04:54:40PM -0400, Tom Lane wrote: > >> This works about 99% of the time, in fact. It falls down in the --clean > > > Note that this fails for me (sometimes) even without --clean. > > Oh

Re: Stale external URL in doc?

2020-07-10 Thread Tom Lane
Thomas Munro writes: > The Microsoft one is OK, it's a redirect, but the redirect target > looks like a more permanent URL to me so maybe we should change it. +1 > The others required minor manual sleuthing to correct; I hope I found > the correct ISN ranges page. Please see attached. I didn't

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Fri, Jul 10, 2020 at 7:17 AM Stephen Frost wrote: > > > The hash_mem design (as it stands) would affect both hash join and > > > hash aggregate. I believe that it makes most sense to have hash-based > > > nodes under the control of a single

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Peter Geoghegan writes: > > It now seems likely that the hash_mem/hash_mem_multiplier proposal has > > the support it needs to get into Postgres 13. Assuming that the > > proposal doesn't lose momentum, then it's about time to return to the > >

Re: Stale external URL in doc?

2020-07-10 Thread Daniel Gustafsson
> On 10 Jul 2020, at 23:47, Tom Lane wrote: > Thomas Munro writes: >> The others required minor manual sleuthing to correct; I hope I found >> the correct ISN ranges page. Please see attached. > > I didn't actually check any of these, but they look like sane changes. +1, looks good, thanks!

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Tom Lane
Stephen Frost writes: > I don't see hash_mem as being any kind of proper fix- it's just punting > to the user saying "we can't figure this out, how about you do it" and, > worse, it's in conflict with how we already ask the user that question. > Turning it into a multiplier doesn't change that eit

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2020-07-10 Thread Alvaro Herrera
On 2019-Sep-09, Amit Kapila wrote: > Thanks for picking up this. However, I noticed that previously > Horiguchi-San has given some comments on this patch [1] which doesn't > seem to be addressed or at least not all of them are addressed. It is > possible that you would have already addressed tho

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Peter Geoghegan
On Fri, Jul 10, 2020 at 2:50 PM Stephen Frost wrote: > Nothing of what you've said thus far has shown me that there were > material bits of the discussion that I've missed. Maybe that's just because you missed those bits too? > No, that other people > feel differently or have made comments suppo

Re: min_safe_lsn column in pg_replication_slots view

2020-07-10 Thread Alvaro Herrera
On 2020-Jul-09, Alvaro Herrera wrote: > I think we should define InvalidXLogSegNo to be ~((uint64)0) and add a > macro to test for that. That's overkill really. I just used zero. Running contrib/test_decoding under valgrind, this now passes. I think I'd rather do away with the compare to zero,

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread David Rowley
On Sat, 11 Jul 2020 at 10:00, Tom Lane wrote: > > Stephen Frost writes: > > I don't see hash_mem as being any kind of proper fix- it's just punting > > to the user saying "we can't figure this out, how about you do it" and, > > worse, it's in conflict with how we already ask the user that questio

Re: jsonpath versus NaN

2020-07-10 Thread Alexander Korotkov
On Thu, Jul 9, 2020 at 4:04 AM Alexander Korotkov wrote: > I understand both patches as fixes and propose to backpatch them to 12 > if no objections. Both patches are pushed. -- Regards, Alexander Korotkov

Re: distribute_restrictinfo_to_rels if restrictinfo contains volatile functions

2020-07-10 Thread Zhenghua Lyu
Hi, Thanks for your reply. I find the problem in a distributed database based on Postgres (Greenplum). In distributed database there may be distributed tables: every single node only contain subpart of the data and combine them all will get the full data I think it may

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread David G. Johnston
On Fri, Jul 10, 2020 at 5:16 PM David Rowley wrote: > Stephen mentions in [1] that: > > Users who are actually hit by this in a negative way > > have an option- increase work_mem to reflect what was actually happening > > already. > > Peter is not a fan of that idea, which can only be due to the

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread David Rowley
On Sat, 11 Jul 2020 at 12:47, David G. Johnston wrote: > The multiplier seems strictly better than "rely on work_mem alone, i.e., do > nothing"; the detracting factor being one more GUC. Even if one wants to > argue the solution is ugly or imperfect the current state seems worse and a > more p

Re: pg_regress cleans up tablespace twice.

2020-07-10 Thread Michael Paquier
On Fri, Jul 10, 2020 at 09:35:56AM -0400, Tom Lane wrote: > Should now be possible to undo whatever hack you had to use ... Yes, I have also opened an issue on github: https://github.com/macdice/cfbot/issues/11/ -- Michael signature.asc Description: PGP signature

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread David G. Johnston
On Fri, Jul 10, 2020 at 6:19 PM David Rowley wrote: > If we have to have a new GUC, my preference would be hashagg_mem, > where -1 means use work_mem and a value between 64 and MAX_KILOBYTES > would mean use that value. We'd need some sort of check hook to > disallow 0-63. I really am just faili

Re: Does TupleQueueReaderNext() really need to copy its result?

2020-07-10 Thread Soumyadeep Chakraborty
Hi Thomas, +1 to the idea! I ran some experiments on both of your patches. I could reproduce the speed gain that you saw for a plan with a simple parallel sequential scan. However, I got no gain at all for a parallel hash join and parallel agg query. -

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread David Rowley
On Sat, 11 Jul 2020 at 13:36, David G. Johnston wrote: > If we add a setting that defaults to work_mem then the benefit is severely > reduced. You still have to modify individual queries, but the change can > simply be more targeted than changing work_mem alone. I think the idea is that this i

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread David G. Johnston
On Fri, Jul 10, 2020 at 6:43 PM David Rowley wrote: > On Sat, 11 Jul 2020 at 13:36, David G. Johnston > wrote: > > If we add a setting that defaults to work_mem then the benefit is > severely reduced. You still have to modify individual queries, but the > change can simply be more targeted than

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread Peter Geoghegan
On Fri, Jul 10, 2020 at 6:19 PM David Rowley wrote: > If we get hash_mem > or some variant that is a multiplier of work_mem, then that user is in > exactly the same situation for that plan. i.e there's no ability to > increase the memory allowances for Hash Agg alone. That's true, of course. > I

Re: Does TupleQueueReaderNext() really need to copy its result?

2020-07-10 Thread Thomas Munro
On Sat, Jul 11, 2020 at 1:37 PM Soumyadeep Chakraborty wrote: > +1 to the idea! I ran some experiments on both of your patches. Hi Soumyadeep, Thanks for testing! > I could reproduce the speed gain that you saw for a plan with a simple > parallel sequential scan. However, I got no gain at all f

Re: Stale external URL in doc?

2020-07-10 Thread Thomas Munro
On Sat, Jul 11, 2020 at 9:56 AM Daniel Gustafsson wrote: > > On 10 Jul 2020, at 23:47, Tom Lane wrote: > > Thomas Munro writes: > >> The others required minor manual sleuthing to correct; I hope I found > >> the correct ISN ranges page. Please see attached. > > > > I didn't actually check any o

Re: Towards easier AMs: Cleaning up inappropriate use of name "relkind"

2020-07-10 Thread Michael Paquier
On Wed, Jul 08, 2020 at 10:00:47PM +0900, Michael Paquier wrote: > There are two patches on this thread v2-0001 being much smaller than > v2-0002. I have looked at 0001 for now, and, like Alvaro, this > renaming makes sense to me. Those commands work on objects that are > relkinds, except for one

Re: proposal: schema variables

2020-07-10 Thread Pavel Stehule
po 6. 7. 2020 v 10:17 odesílatel Pavel Stehule napsal: > > > ne 5. 7. 2020 v 15:33 odesílatel Pavel Stehule > napsal: > >> >> >> čt 21. 5. 2020 v 14:49 odesílatel Pavel Stehule >> napsal: >> >>> >>> >>> čt 21. 5. 2020 v 13:34 odesílatel Amit Kapila >>> napsal: >>> On Thu, May 21, 2020 at

Re: Default setting for enable_hashagg_disk

2020-07-10 Thread David Rowley
On Sat, 11 Jul 2020 at 14:02, Peter Geoghegan wrote: > > On Fri, Jul 10, 2020 at 6:19 PM David Rowley wrote: > > If we have to have a new GUC, my preference would be hashagg_mem, > > where -1 means use work_mem and a value between 64 and MAX_KILOBYTES > > would mean use that value. We'd need som

Re: calling procedures is slow and consumes extra much memory against calling function

2020-07-10 Thread Pavel Stehule
čt 9. 7. 2020 v 8:28 odesílatel Amit Khandekar napsal: > On Wed, 17 Jun 2020 at 13:54, Pavel Stehule > wrote: > > > > > > > > st 17. 6. 2020 v 7:52 odesílatel Amit Khandekar > napsal: > >> > >> On Wed, 10 Jun 2020 at 17:12, Pavel Stehule > wrote: > >> > st 10. 6. 2020 v 12:26 odesílatel Amit K

Re: Towards easier AMs: Cleaning up inappropriate use of name "relkind"

2020-07-10 Thread Michael Paquier
On Wed, Jul 01, 2020 at 05:04:19PM -0700, Mark Dilger wrote: > Most of the work in this patch is mechanical replacement of if/else > if/else statements which hinge on relkind to switch statements on > relkind. The patch is not philosophically very interesting, but it > is fairly long. Reviewers m

Re: pg_regress cleans up tablespace twice.

2020-07-10 Thread Michael Paquier
On Sat, Jul 11, 2020 at 10:35:07AM +0900, Michael Paquier wrote: > Yes, I have also opened an issue on github: > https://github.com/macdice/cfbot/issues/11/ And Thomas has just fixed it: https://github.com/macdice/cfbot/commit/e78438444a00bc8d83863645503b2f7c1a9da016 -- Michael signature.asc Des