Re: Using XLogFileNameP in critical section

2019-12-01 Thread Michael Paquier
On Fri, Nov 29, 2019 at 06:44:58PM +0100, Masahiko Sawada wrote: > I encountered that the assertion error is reported instead of a proper > PANIC message when failed to fsync WAL. The cause is that there are > multiple places where we call XLogFileNameP function that calls palloc > during critical

Re: Implementing Incremental View Maintenance

2019-12-01 Thread Yugo Nagata
On Mon, 02 Dec 2019 10:36:36 +0900 (JST) Tatsuo Ishii wrote: > >> One thing pending in this development line is how to catalogue aggregate > >> functions that can be used in incrementally-maintainable views. > >> I saw a brief mention somewhere that the devels knew it needed to be > >> done, but

Re: Control your disk usage in PG: Introduction to Disk Quota Extension

2019-12-01 Thread Haozhou Wang
Hi Michael, Thank you very much for your email. I rebased the code with the newest master and attached it in the attachment. Thank you very much! Regards, Haozhou On Sun, Dec 1, 2019 at 11:20 AM Michael Paquier wrote: > On Fri, Sep 27, 2019 at 11:30:08AM +0800, Haozhou Wang wrote: > > I

Re: pgbench -i progress output on terminal

2019-12-01 Thread Michael Paquier
On Mon, Dec 02, 2019 at 02:30:47PM +0900, Amit Langote wrote: > On Sun, Dec 1, 2019 at 4:33 AM Fabien COELHO wrote: >> Patch applies, compiles, works for me. No further comments. >> >> I switched the patch as ready. > > Thanks a lot. An issue with the patch as proposed is that its style is

Re: Add a GUC variable that control logical replication

2019-12-01 Thread Craig Ringer
On Thu, 28 Nov 2019 at 11:53, Michael Paquier wrote: > On Wed, Nov 06, 2019 at 10:01:43PM +0800, Quan Zongliang wrote: > > What the user needs is the same replication link that selectively skips > some > > transactions. And this choice only affects transactions that are doing > bulk > > delete

Re: Missing data_sync_elevel() for some calls of pg_fsync()?

2019-12-01 Thread Craig Ringer
On Mon, 2 Dec 2019 at 13:43, Thomas Munro wrote: > > 1. Some code opens a file, writes some stuff to it, closes, and then > fsyncs it, and if that fails and and it ever retries it'll redo all of > those steps again. We know that some kernels might have thrown away > the data, but we don't care

Re: Missing data_sync_elevel() for some calls of pg_fsync()?

2019-12-01 Thread Thomas Munro
On Mon, Dec 2, 2019 at 5:58 PM Michael Paquier wrote: > I was just looking at some callers of pg_fsync(), to notice that some > code paths don't use data_sync_elevel(). For some code paths, that's > actually better to never PANIC (say backup_label file, logical > decoding snapshot, lock file

Re: pgbench -i progress output on terminal

2019-12-01 Thread Amit Langote
Hi Fabien, On Sun, Dec 1, 2019 at 4:33 AM Fabien COELHO wrote: > Patch applies, compiles, works for me. No further comments. > > I switched the patch as ready. Thanks a lot. Regards, Amit

Missing data_sync_elevel() for some calls of pg_fsync()?

2019-12-01 Thread Michael Paquier
Hi all, I was just looking at some callers of pg_fsync(), to notice that some code paths don't use data_sync_elevel(). For some code paths, that's actually better to never PANIC (say backup_label file, logical decoding snapshot, lock file where FATAL/LOG are used now, etc.). However I have

Re:Issue about memory order on ARM

2019-12-01 Thread 盏一
Sorry to bother you, now I know that there is no problem here. The model for reading and writing of PGXACT::xid and ShmemVariableCache->latestCompletedXid can be simplified as follows: backend A backend B backend C wlock(XidGenLock);

Failure in TAP tests of pg_ctl on Windows with parallel instance set

2019-12-01 Thread Michael Paquier
Hi all, I have run the TAP tests with an instance of Postgres locally set at port 5432 on Windows, to notice that 001_start_stop.pl fails various tests because the test tries to use the default port for the node initialized with pg_ctl. The problem can get fixed easily by assigning a random port

Re: Bogus EXPLAIN results with column aliases for mismatched partitions

2019-12-01 Thread Amit Langote
On Sun, Dec 1, 2019 at 11:40 AM Tom Lane wrote: > then the EXPLAIN output produced by HEAD looks like: > > QUERY PLAN > --- > Sort >Output: p.x, p.b >Sort Key: p.x >-> Append > -> Seq Scan on public.part_p1 p >

Re: fe-utils - share query cancellation code

2019-12-01 Thread Michael Paquier
On Fri, Nov 29, 2019 at 08:44:25AM +0100, Fabien COELHO wrote: > I do not have a Windows host, so I can only do blind tests. I just moved the > declaration out of !WIN32 scope in attached v7, which might solve the > resolution error. All other issues pointed out above seem fixed in the v6 > you

Re: Implementing Incremental View Maintenance

2019-12-01 Thread Takuma Hoshiai
On Mon, 02 Dec 2019 10:01:18 +0900 (JST) Tatsuo Ishii wrote: > Michael, > > > A rebase looks to be necessary, Mr Robot complains that the patch does > > not apply cleanly. As the thread is active recently, I have moved the > > patch to next CF, waiting on author. > > Thank you for taking care

RE: extension patch of CREATE OR REPLACE TRIGGER

2019-12-01 Thread osumi.takami...@fujitsu.com
Dear Michael san > The latest patch includes calls to heap_open(), causing its compilation to > fail. > Could you please send a rebased version of the patch? I have moved the entry > to > next CF, waiting on author. Thanks. I've fixed where you pointed out. Also, I'm waiting for other kind of

Re: Implementing Incremental View Maintenance

2019-12-01 Thread Tatsuo Ishii
> On Fri, Nov 29, 2019 at 06:19:54PM +0900, Yugo Nagata wrote: >> Sorry, an unfinished line was left... Please ignore this. > > A rebase looks to be necessary, Mr Robot complains that the patch does > not apply cleanly. Is this because the patch has ".gz" suffix? Best regards, -- Tatsuo Ishii

Re: Yet another vectorized engine

2019-12-01 Thread Hubert Zhang
On Sun, Dec 1, 2019 at 10:05 AM Michael Paquier wrote: > On Thu, Nov 28, 2019 at 05:23:59PM +0800, Hubert Zhang wrote: > > Note that the vectorized executor engine is based on PG9.6 now, but it > > could be ported to master / zedstore with some effort. We would > appreciate > > some feedback

Re: Implementing Incremental View Maintenance

2019-12-01 Thread Tatsuo Ishii
>> One thing pending in this development line is how to catalogue aggregate >> functions that can be used in incrementally-maintainable views. >> I saw a brief mention somewhere that the devels knew it needed to be >> done, but I don't see in the thread that they got around to doing it. >> Did you

Re: Yet another vectorized engine

2019-12-01 Thread Hubert Zhang
Hi Konstantin, Thanks for your reply. On Fri, Nov 29, 2019 at 12:09 AM Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > On 28.11.2019 12:23, Hubert Zhang wrote: > > We just want to introduce another POC for vectorized execution engine > https://github.com/zhangh43/vectorize_engine and

Re: Implementing Incremental View Maintenance

2019-12-01 Thread Tatsuo Ishii
Michael, > A rebase looks to be necessary, Mr Robot complains that the patch does > not apply cleanly. As the thread is active recently, I have moved the > patch to next CF, waiting on author. Thank you for taking care of this patch. Hoshiai-san, can you please rebase the patch? Best regards,

Re: Allow 'sslkey' and 'sslcert' in postgres_fdw user mappings

2019-12-01 Thread Andrew Dunstan
On 11/30/19 8:48 PM, Michael Paquier wrote: > On Thu, Oct 31, 2019 at 07:54:41PM -0400, Andrew Dunstan wrote: >> This patch achieves $SUBJECT and also provides some testing of the >> sslpassword setting. > The patch does not apply anymore, so a rebase is needed. As it has > not been reviewed, I

Re: Bogus EXPLAIN results with column aliases for mismatched partitions

2019-12-01 Thread Tom Lane
I wrote: > Now, there is another thing that set_rtable_names() is doing that > postgres_fdw doesn't do, which is to unique-ify the chosen alias > names by adding "_NN" if the querytree contains multiple uses of > the same table alias. I don't see any practical way for postgres_fdw > to match that

Re: Bogus EXPLAIN results with column aliases for mismatched partitions

2019-12-01 Thread Tom Lane
I wrote: > * variant-a's diffs in expected/postgres_fdw.out indicate that > postgres_fdw is doing something weird with the table aliases it selects to > print in the "Relations:" output. I think this is an independent bug --- > and it surely *is* a bug, because the aliases printed by HEAD don't

Re: surprisingly expensive join planning query

2019-12-01 Thread Tom Lane
Tomas Vondra writes: > On Sun, Dec 01, 2019 at 01:27:04PM -0500, Tom Lane wrote: >> Alternatively, it'd be possible to get rid of the separate List >> altogether, and just add the rinfo's to "mergeclauses" immediately. >> The functionality of the separate list could be replaced by a >> bool

Re: Using multiple extended statistics for estimates

2019-12-01 Thread Tomas Vondra
On Sat, Nov 30, 2019 at 03:01:31PM -0800, Mark Dilger wrote: Are you planning to submit a revised patch for this? Yes, I'll submit a rebased version of this patch shortly. I got broken because of the recent fix in choose_best_statistics, shouldn't take long to update the patch. I do have a

Re: surprisingly expensive join planning query

2019-12-01 Thread Tomas Vondra
On Sun, Dec 01, 2019 at 01:27:04PM -0500, Tom Lane wrote: Tomas Vondra writes: It seems most of this comesfrom find_mergeclauses_for_outer_pathkeys() which builds matched_restrictinfos and then just leaves it allocated. After pfreeing this (see attached patch), the memory usage gets way down

Re: surprisingly expensive join planning query

2019-12-01 Thread Tom Lane
Tomas Vondra writes: > It seems most of this comesfrom find_mergeclauses_for_outer_pathkeys() > which builds matched_restrictinfos and then just leaves it allocated. > After pfreeing this (see attached patch), the memory usage gets way down > and the query completes. Interesting. The memory

Re: [HACKERS] Block level parallel vacuum

2019-12-01 Thread Sergei Kornilov
Hi > I think I got your point. Your proposal is that it's more efficient if > we make the leader process vacuum the index that can be processed only > the leader process (i.e. indexes not supporting parallel index vacuum) > while workers are processing indexes supporting parallel index vacuum, >

surprisingly expensive join planning query

2019-12-01 Thread Tomas Vondra
Hi, while evaluating one of the CF patches (the incremental sort one, but that's mostly irrelevant), I ran into a strange issue with join planning for a fairly simple query. I needed to asses how the patch affects query planning for different GUCs, so I ran a group of queries and stashed the

Re: [HACKERS] Block level parallel vacuum

2019-12-01 Thread Masahiko Sawada
On Sat, 30 Nov 2019 at 22:11, Mahendra Singh wrote: > > On Sat, 30 Nov 2019 at 19:18, Sergei Kornilov wrote: >> >> Hello >> >> Its possible to change order of index processing by parallel leader? In v35 >> patchset I see following order: >> - start parallel processes >> - leader and parallel

Re: bitmaps and correlation

2019-12-01 Thread Justin Pryzby
On Sun, Dec 01, 2019 at 12:34:37PM +0900, Michael Paquier wrote: > On Sat, Nov 02, 2019 at 03:26:17PM -0500, Justin Pryzby wrote: > > Attached is a fixed and rebasified patch for cfbot. > > Included inline for conceptual review. > > Your patch still causes two regression tests to fail per Mr

Re: [HACKERS] Block level parallel vacuum

2019-12-01 Thread Masahiko Sawada
On Sat, 30 Nov 2019 at 04:06, Amit Kapila wrote: > > On Fri, Nov 29, 2019 at 7:11 PM Masahiko Sawada > wrote: > > > > On Thu, 28 Nov 2019 at 11:57, Amit Kapila wrote: > > > > > > > > I think it is probably because this part of the code doesn't consider > > >

Re: [HACKERS] Block level parallel vacuum

2019-12-01 Thread Masahiko Sawada
On Sun, 1 Dec 2019 at 11:06, Sergei Kornilov wrote: > > Hi > > > I think the advantage of the current approach is that once the parallel > > workers are launched, the leader can process indexes that don't support > > parallelism. So, both type of indexes can be processed at the same time. > >

Re: Issue about memory order on ARM

2019-12-01 Thread Tom Lane
"=?utf-8?B?55uP5LiA?=" writes: > The code in GetSnapshotData() that read the `xid` field of PGXACT > struct has a dependency on code in GetNewTransactionId() that write > `MyPgXact-xid`. It means that the store of xid should happen before the > load of it. In C11, we can use [Release-Acquire

Re: pg_upgrade fails with non-standard ACL

2019-12-01 Thread Grigory Smolkin
Hello! On 11/29/19 11:07 AM, Artur Zakirov wrote: If Anastasia doesn't mind I'd like to send new version of the patch. On 2019/11/28 12:29, Artur Zakirov wrote: On 2019/11/27 13:22, Michael Paquier wrote: Yeah, the actual take is if we want to make the frontend code more complicated with a

Re: [HACKERS] Block level parallel vacuum

2019-12-01 Thread Sergei Kornilov
Hi > I think the advantage of the current approach is that once the parallel > workers are launched, the leader can process indexes that don't support > parallelism.  So, both type of indexes can be processed at the same time. In lazy_parallel_vacuum_or_cleanup_indexes I see: /*