Re: Wait event that should be reported while waiting for WAL archiving to finish

2020-02-16 Thread Fujii Masao
On 2020/02/14 23:43, Robert Haas wrote: On Thu, Feb 13, 2020 at 10:47 PM Fujii Masao wrote: Fixed. Thanks for the review! I think it would be safer to just report the wait event during pg_usleep(100L) rather than putting those calls around the whole loop. It does not seem impossible

Re: Wait event that should be reported while waiting for WAL archiving to finish

2020-02-16 Thread Fujii Masao
On 2020/02/14 15:45, Michael Paquier wrote: On Fri, Feb 14, 2020 at 12:47:19PM +0900, Fujii Masao wrote: logical_rewrite_truncate_v1.patch adds the description of LogicalRewriteTruncate into the doc. This needs to be back-patched to v10 where commit 249cf070e3 introduced

RE: [PoC] Non-volatile WAL buffer

2020-02-16 Thread Takashi Menjo
Hello Amit, > I apologize for not having any opinion on the patches themselves, but let me > point out that it's better to base these > patches on HEAD (master branch) than REL_12_0, because all new code is > committed to the master branch, > whereas stable branches such as REL_12_0 only

Re: Conflict handling for COPY FROM

2020-02-16 Thread Tatsuo Ishii
>> test=# copy t1 from '/tmp/a' with (error_limit 1); >> ERROR: duplicate key value violates unique constraint "t1_pkey" >> DETAIL: Key (i)=(2) already exists. >> CONTEXT: COPY t1, line 2: "2 2" >> >> So if the number of errors raised exceeds error_limit, no constaraint >> violating rows (in

tiny documentation fix

2020-02-16 Thread Amit Langote
Hi, I propose this small fix for 27.4. Progress Reporting: -all of its partitions are also recursively analyzed as also mentioned on +all of its partitions are also recursively analyzed as also mentioned in . Note the last word: "in" sounds more correct. Thanks, Amit diff --git

Re: Conflict handling for COPY FROM

2020-02-16 Thread Surafel Temesgen
Hi, > > ERROR_LIMIT ' class="parameter">limit_number' > > > > I think this should be: > > > > ERROR_LIMIT limit_number > > > > (no single quote) > > Thank you .Fixed > More comments: > > - I think the document should stat that if limit_number = 0, all > errors are immediately raised

Re: error context for vacuum to include block number

2020-02-16 Thread Justin Pryzby
On Mon, Feb 17, 2020 at 02:18:11PM +0900, Masahiko Sawada wrote: > Oops it seems to me that it's better to set error context after > tupindex = 0. Sorry for my bad. I take your point but did it differently - see what you think > And the above comment can be written in a single line as other same

Re: ALTER tbl rewrite loses CLUSTER ON index (consider moving indisclustered to pg_class)

2020-02-16 Thread Justin Pryzby
On Mon, Feb 17, 2020 at 02:31:42PM +0900, Amit Langote wrote: > Hi Justin, > > On Fri, Feb 7, 2020 at 11:39 PM Justin Pryzby wrote: > > On Thu, Feb 06, 2020 at 02:24:47PM -0300, Alvaro Herrera wrote: > > > On 2020-Feb-06, Justin Pryzby wrote: > > > > > > > I wondered if it wouldn't be better if

Re: ALTER tbl rewrite loses CLUSTER ON index (consider moving indisclustered to pg_class)

2020-02-16 Thread Amit Langote
Hi Justin, On Fri, Feb 7, 2020 at 11:39 PM Justin Pryzby wrote: > On Thu, Feb 06, 2020 at 02:24:47PM -0300, Alvaro Herrera wrote: > > On 2020-Feb-06, Justin Pryzby wrote: > > > > > I wondered if it wouldn't be better if CLUSTER ON was stored in pg_class > > > as the > > > Oid of a clustered

Re: small improvement of the elapsed time for truncating heap in vacuum

2020-02-16 Thread Kasahara Tatsuhito
Hi, On Mon, Feb 17, 2020 at 1:07 PM Masahiko Sawada wrote: > For the patch, we can put also the declaration of ru0 into the loop. Thanks for your reply. Hmm, certainly that it may be better. Fix the v2 patch and attached. Best regards, -- Tatsuhito Kasahara kasahara.tatsuhito _at_ gmail.com

Re: error context for vacuum to include block number

2020-02-16 Thread Masahiko Sawada
On Mon, 17 Feb 2020 at 12:57, Justin Pryzby wrote: > > On Mon, Feb 17, 2020 at 10:47:47AM +0900, Masahiko Sawada wrote: > > Thank you for updating the patch! > > Thank you for updating the patch. > > 1. > > The above lines need a new line. > > Done, thanks. > > > 2. > > In lazy_vacuum_heap, we

Re: Collation versioning

2020-02-16 Thread Thomas Munro
On Thu, Feb 13, 2020 at 8:13 AM Julien Rouhaud wrote: > Hearing no complaints on the suggestions, I'm attaching v8 to address that: > > - pg_dump is now using a binary_upgrade_set_index_coll_version() function > rather than plain DDL > - the additional DDL is now of the form: > ALTER INDEX

Re: assert pg_class.relnatts is consistent

2020-02-16 Thread Justin Pryzby
On Mon, Feb 17, 2020 at 01:25:05PM +0900, Amit Langote wrote: > > Pushed both of those. > > Thank you. > > It's amazing to see how simple bootstrapping has now become thanks to > the work you guys have done recently. On Fri, Feb 14, 2020 at 06:00:05PM +0900, Amit Langote wrote: > > I can't

Re: [PoC] Non-volatile WAL buffer

2020-02-16 Thread Amit Langote
Menjo-san, On Mon, Feb 17, 2020 at 1:13 PM Takashi Menjo wrote: > I applied my patchset that mmap()-s WAL segments as WAL buffers to > refs/tags/REL_12_0, and measured and analyzed its performance with pgbench. > Roughly speaking, When I used *SSD and ext4* to store WAL, it was "obviously >

Re: assert pg_class.relnatts is consistent

2020-02-16 Thread Amit Langote
On Sun, Feb 16, 2020 at 5:25 AM Tom Lane wrote: > I wrote: > > So that leads me to the attached. > > ... > > (I agree with Alvaro's thought of shortening AddDefaultValues, > > but didn't do that here.) > > Pushed both of those. Thank you. It's amazing to see how simple bootstrapping has now

Re: Standards compliance of SET ROLE / SET SESSION AUTHORIZATION

2020-02-16 Thread Craig Ringer
On Sat, 15 Feb 2020 at 05:36, Tom Lane wrote: > > Chapman Flack writes: > > On 2/14/20 4:01 PM, Tom Lane wrote: > >> ... A protocol-level message > >> to set session auth could also be possible, of course. > > > I'll once again whimper softly and perhaps ineffectually that an > > SQL-exposed

RE: [PoC] Non-volatile WAL buffer

2020-02-16 Thread Takashi Menjo
Dear hackers, I applied my patchset that mmap()-s WAL segments as WAL buffers to refs/tags/REL_12_0, and measured and analyzed its performance with pgbench. Roughly speaking, When I used *SSD and ext4* to store WAL, it was "obviously worse" than the original REL_12_0. VTune told me that the

Re: small improvement of the elapsed time for truncating heap in vacuum

2020-02-16 Thread Masahiko Sawada
On Mon, 17 Feb 2020 at 12:44, Kasahara Tatsuhito wrote: > > Hi, > > On Fri, Feb 14, 2020 at 4:50 PM Fujii Masao wrote: > > Regarding the patch, isn't it better to put pg_rusage_init() at the > > top of do loop block? If we do this, as a side-effect, we can get > > rid of pg_rusage_init() at the

Re: error context for vacuum to include block number

2020-02-16 Thread Justin Pryzby
On Mon, Feb 17, 2020 at 10:47:47AM +0900, Masahiko Sawada wrote: > Thank you for updating the patch! > > 1. > The above lines need a new line. Done, thanks. > 2. > In lazy_vacuum_heap, we set the error context and then call > pg_rusage_init whereas lazy_vacuum_index and lazy_cleanup_index does

Re: small improvement of the elapsed time for truncating heap in vacuum

2020-02-16 Thread Kasahara Tatsuhito
Hi, On Fri, Feb 14, 2020 at 4:50 PM Fujii Masao wrote: > Regarding the patch, isn't it better to put pg_rusage_init() at the > top of do loop block? If we do this, as a side-effect, we can get > rid of pg_rusage_init() at the top of lazy_truncate_heap(). Thanks for your reply. Yeah, it makes

Re: jsonb_object() seems to be buggy. jsonb_build_object() is good.

2020-02-16 Thread Bryn Llewellyn
On 16-Feb-2020, at 16:40, Andrew Dunstan wrote: On 2/16/20 7:25 PM, Bryn Llewellyn wrote: > > B.t.w., you earlier said “The double quotes [around “dog”] serve a specific > purpose, to allow values containing commas to be treated as a single value > (see syntax details for the exact rules) in

Re: Conflict handling for COPY FROM

2020-02-16 Thread Tatsuo Ishii
> In your patch for copy.sgml: > > ERROR_LIMIT 'limit_number' > > I think this should be: > > ERROR_LIMIT limit_number > > (no single quote) More comments: - I think the document should stat that if limit_number = 0, all errors are immediately raised (behaves same as current

Flexible pglz_stategy values and delete const.

2020-02-16 Thread Moon, Insung
Dear Hackers. For the current PostgreSQL, the toast data and some WAL data (page data) are compressed using the LZ-based algorithm. For PostgreSQL, two types of strategy(PGLZ_strategy_default, PGLZ_strategy_always) are provided by default, and only used of PGLZ_strategy_default(default values) on

Re: 1 Status of vertical clustered index - 2 Join using (fk_constraint) suggestion - 3 Status of pgsql's parser autonomization

2020-02-16 Thread maxzor
... Thank you will look into it! I'm not really sure what's the point / benefit here. Initially it seemed you simply propose a syntax saying "do a join using the columns in the FK constraint" but it's unclear to me how this implies any writing speedup? This is exactly what I mean. If you

Re: error context for vacuum to include block number

2020-02-16 Thread Masahiko Sawada
On Sat, 15 Feb 2020 at 00:34, Justin Pryzby wrote: > > On Fri, Feb 14, 2020 at 12:30:25PM +0900, Masahiko Sawada wrote: > > * I think the function name is too generic. init_vacuum_error_callback > > or init_vacuum_errcallback is better. > > > * The comment of this function is not accurate since

Re: 1 Status of vertical clustered index - 2 Join using (fk_constraint) suggestion - 3 Status of pgsql's parser autonomization

2020-02-16 Thread Tomas Vondra
On Sun, Feb 16, 2020 at 10:38:29PM +0100, maxzor wrote: Hello, 1. I was told that M$ SQLServer provides huge performance deltas over PostgreSQL when dealing with index-unaligned queries : create index i on t (a,b, c); select * from t where b=... and c=...; Perhaps index-only scans might

Re: jsonb_object() seems to be buggy. jsonb_build_object() is good.

2020-02-16 Thread Andrew Dunstan
On 2/16/20 7:25 PM, Bryn Llewellyn wrote: > > B.t.w., you earlier said “The double quotes [around “dog”] serve a specific > purpose, to allow values containing commas to be treated as a single value > (see syntax details for the exact rules) in the resulting array of text > values.” But this

Re: jsonb_object() seems to be buggy. jsonb_build_object() is good.

2020-02-16 Thread Bryn Llewellyn
Andrew Dunstan wrote: Bryn Llewellyn wrote: > > Andrew replied > > The function above has many deficiencies, including lack of error > checking and use of 'execute' which will significantly affect > performance. Still, if it works for you, that's your affair. > > These functions were written

Re: pgindent && weirdness

2020-02-16 Thread Thomas Munro
On Fri, Jan 17, 2020 at 3:58 PM Tom Lane wrote: > Alvaro Herrera writes: > > On 2020-Jan-16, Tom Lane wrote: > >> ... But I was hoping to > >> hear Piotr's opinion before moving forward. Me too. Thinking about this again: It's obviously not true that everything that looks like a function call

1 Status of vertical clustered index - 2 Join using (fk_constraint) suggestion - 3 Status of pgsql's parser autonomization

2020-02-16 Thread maxzor
Hello, 1. I was told that M$ SQLServer provides huge performance deltas over PostgreSQL when dealing with index-unaligned queries : create index i on t (a,b, c); select * from t where b=... and c=...; Columnar storage has been tried by various companies, CitusData, EnterpriseDB, 2ndQuadrant,

Re: jsonb_object() seems to be buggy. jsonb_build_object() is good.

2020-02-16 Thread Andrew Dunstan
On 2/16/20 1:40 PM, Bryn Llewellyn wrote: > > Andrew replied > > The function above has many deficiencies, including lack of error > checking and use of 'execute' which will significantly affect > performance. Still, if it works for you, that's your affair. > > These functions were written to

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-02-16 Thread Tom Lane
Andres Freund writes: > On 2020-02-14 13:34:03 -0500, Robert Haas wrote: >> I think the group locking + deadlock detection things are more >> fundamental than you might be crediting, but I agree that having >> parallel mechanisms has its own set of pitfalls. > It's possible. But I'm also

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2020-02-16 Thread Andres Freund
Hi, On 2020-02-14 13:34:03 -0500, Robert Haas wrote: > On Fri, Feb 14, 2020 at 1:07 PM Andres Freund wrote: > > Yea, that seems possible. I'm not really sure it's needed however? As > > long as you're not teaching the locking mechanism new tricks that > > influence the wait graph, why would the

reindex concurrently and two toast indexes

2020-02-16 Thread Justin Pryzby
Forking old, long thread: https://www.postgresql.org/message-id/36712441546604286%40sas1-890ba5c2334a.qloud-c.yandex.net On Fri, Jan 04, 2019 at 03:18:06PM +0300, Sergei Kornilov wrote: > About reindex invalid indexes - i found one good question in archives [1]: > how about toast indexes? > I

Re: jsonb_object() seems to be buggy. jsonb_build_object() is good.

2020-02-16 Thread Bryn Llewellyn
Bryn Llewellyn wrote: > ...I wrote my own wrapper for jsonb_build_array() > and jsonb_build_object(): > > create function my_jsonb_build( > kind in varchar, > variadic_elements in varchar) > returns jsonb > immutable > language plpgsql > as $body$ > declare > stmt varchar := >

Re: explain HashAggregate to report bucket and memory stats

2020-02-16 Thread Justin Pryzby
Updated: . remove from explain analyze those tests which would display sort Memory/Disk. Oops. . fix issue with the first patch showing zero "tuples" memory for some grouping sets. . reclassify memory as "tuples" if it has to do with "members". So hashtable size is now redundant

Re: plan cache overhead on plpgsql expression

2020-02-16 Thread Pavel Stehule
ne 16. 2. 2020 v 15:12 odesílatel Pavel Stehule napsal: > Hi > > when I do some profiling of plpgsql, usually I surprised how significant > overhead has expression execution. Any calculations are very slow. > > This is not typical example of plpgsql, but it shows cleanly where is a > overhead >

Re: [Proposal] Global temporary tables

2020-02-16 Thread Pavel Stehule
ne 16. 2. 2020 v 16:15 odesílatel 曾文旌(义从) napsal: > > > 2020年2月15日 下午6:06,Pavel Stehule 写道: > > > postgres=# insert into foo select generate_series(1,1); >> INSERT 0 1 >> postgres=# \dt+ foo >> List of relations >>

Re: [Proposal] Global temporary tables

2020-02-16 Thread 曾文旌(义从)
> 2020年2月15日 下午6:06,Pavel Stehule 写道: > > >> postgres=# insert into foo select generate_series(1,1); >> INSERT 0 1 >> postgres=# \dt+ foo >> List of relations >> ┌┬──┬───┬───┬─┬┬─┐ >> │ Schema │ Name │ Type

plan cache overhead on plpgsql expression

2020-02-16 Thread Pavel Stehule
Hi when I do some profiling of plpgsql, usually I surprised how significant overhead has expression execution. Any calculations are very slow. This is not typical example of plpgsql, but it shows cleanly where is a overhead CREATE OR REPLACE FUNCTION public.foo() RETURNS void LANGUAGE plpgsql