Speed up build on Windows by generating symbol definition in batch

2019-03-29 Thread Peifeng Qiu
Hi, hackers. Build process on Windows includes compiling all source into object files, linking them to binaries, and generating export symbol definitions, etc. When I watched the whole build process with a task manager, I discovered that a lot of time was spent on generating export symbol definiti

Column lookup in a row performance

2019-03-29 Thread Павлухин Иван
Hi PostgresSQL developers, I asked my question already on pgsql-general list and did not find an explanation. Below is the question mainly copied from [0]. I am learning deeply how tuples are organized and column values are accessed in different databases. As far as undertood postgres does n

Indexscan failed assert caused by using index without lock

2019-03-29 Thread 高增琦
Following example can reproduce the problem: ``` create table d(a int); create index di on d(a); set enable_seqscan=off; set enable_bitmapscan to off; prepare p as delete from d where a=3; execute p; execute p; ``` The reason is that: ExecInitIndexScan will not lock index because it thinks InitPl

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Robert Haas wrote: > On Fri, Mar 29, 2019 at 3:28 PM Alvaro Herrera > wrote: > > Maybe we can consider using dynamic shmem for that, and include a > > pointer to it in the fixed-size chunk. (It's a bit too late to be > > writing this code, mind; I'm just proposing this for a fu

Re: pgbench - add pseudo-random permutation function

2019-03-29 Thread Hironobu SUZUKI
On 2019/03/21 17:27, David Steele wrote: Hi Hironobu, Sorry for the late reply. I reviewed this patch. Function nbits(), which was previously discussed, has been simplified by using the function pg_popcount64(). By adding the mathematical explanation, it has been easier to understand the

Re: speeding up planning with partitions

2019-03-29 Thread Amit Langote
Thanks for the new patches. On Sat, Mar 30, 2019 at 9:17 AM Tom Lane wrote: > > Amit Langote writes: > > On 2019/03/29 7:38, Tom Lane wrote: > >> 2. I seriously dislike what's been done in joinrels.c, too. That > >> really seems like a kluge (and I haven't had time to study it > >> closely). >

Re: clean up pg_checksums.sgml

2019-03-29 Thread Michael Paquier
On Fri, Mar 29, 2019 at 09:32:10AM -0500, Justin Pryzby wrote: > PFA patch with minor improvements to documentation. Patch does not apply, and I have reworded the last paragraph about failures while operating. > Also, what do you think about changing user-facing language from > "check checksum" t

Re: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Michael Paquier
On Fri, Mar 29, 2019 at 08:48:03AM -0700, Andres Freund wrote: > Yes, it increases the total runtime quite considerably. And it adds new > failure modes with partially built invalid indexes hanging around that > need to be dropped manually. On top of that CONCURRENTLY needs multiple transactions t

Re: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Michael Paquier
On Fri, Mar 29, 2019 at 03:10:23PM +, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote: > I tried this great feature for partition index. > The first time the REINDEX TABLE CONCURRENTLY statement is executed > to the partition, then an error occurs. Yes, that's a problem. I am adding an op

Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru

2019-03-29 Thread Michael Paquier
On Fri, Mar 29, 2019 at 11:22:55AM -0300, Alvaro Herrera wrote: > Yeah, that looks good to me too. I wonder if we really need it as LOG > though; we don't say anything for actions unless they take more than the > min duration, so why say something for a no-op that takes almost no time? > Maybe mak

Re: Unix socket dir, an idea

2019-03-29 Thread Tom Lane
Danylo Hlynskyi writes: > The problem (as I see it) is that everybody would like to move `/tmp` > socket dir to `/var/run`, or even `/var/run/postgresql` (or even > `/run/postgresql`), but compatibility with old clients (which connect to > /tmp by default) is a concern. *Some* people would like t

Re: speeding up planning with partitions

2019-03-29 Thread Tom Lane
Amit Langote writes: > On 2019/03/29 7:38, Tom Lane wrote: >> 2. I seriously dislike what's been done in joinrels.c, too. That >> really seems like a kluge (and I haven't had time to study it >> closely). > Those hunks account for the fact that pruned partitions, for which we no > longer create

Re: patch to allow disable of WAL recycling

2019-03-29 Thread Jerry Jelinek
On Thu, Mar 28, 2019 at 6:10 PM Thomas Munro wrote: > On Fri, Mar 29, 2019 at 10:47 AM Thomas Munro > wrote: > > On Fri, Mar 29, 2019 at 8:59 AM Robert Haas > wrote: > > > On Tue, Mar 26, 2019 at 3:24 PM Jerry Jelinek < > jerry.jeli...@joyent.com> wrote: > > > > The latest patch is rebased, bui

Re: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Peter Eisentraut
On 2019-03-29 17:01, Justin Pryzby wrote: > On Fri, Mar 29, 2019 at 03:53:05PM +, Bossart, Nathan wrote: >> I noticed a very small typo in the documentation for this feature. > > I submit a bunch more changes for consideration, attached. fixed, thanks -- Peter Eisentraut http:/

Re: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Peter Eisentraut
On 2019-03-29 16:53, Bossart, Nathan wrote: > I noticed a very small typo in the documentation for this feature. fixed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Online verification of checksums

2019-03-29 Thread Michael Banck
Hi, Am Freitag, den 29.03.2019, 16:52 +0100 schrieb Magnus Hagander: > On Fri, Mar 29, 2019 at 4:30 PM Stephen Frost wrote: > > * Magnus Hagander (mag...@hagander.net) wrote: > > > On Thu, Mar 28, 2019 at 10:19 PM Tomas Vondra > > > > > > wrote: > > > > On Thu, Mar 28, 2019 at 01:11:40PM -0700,

Re: explain plans with information about (modified) gucs

2019-03-29 Thread Tomas Vondra
On Wed, Mar 27, 2019 at 09:06:04AM +0100, Rafia Sabih wrote: On Tue, 26 Mar 2019 at 21:04, Tomas Vondra wrote: On Mon, Mar 18, 2019 at 11:31:48AM +0100, Rafia Sabih wrote: >On Sun, 24 Feb 2019 at 00:06, Tomas Vondra wrote: >> >> Hi, >> >> attached is an updated patch, fixing and slightly twea

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Andrew Dunstan
On 3/29/19 11:41 AM, Tom Lane wrote: > Christoph Berg writes: >> What might possibly make sense is to add options to psql to >> facilitate common tasks: >> psql --createdb foo >> psql --createuser bar --superuser >> psql --reindex foo > That's a thought. Or perhaps better, allow pg_ctl to grow

Unix socket dir, an idea

2019-03-29 Thread Danylo Hlynskyi
Before writing a patch, I'd like to hear discussion first. I've searched archives first and read following previous discussions on this topic: - https://www.postgresql.org/message-id/4FCF6040.5030408%40redhat.com - https://www.postgresql.org/message-id/14899.974513046%40sss.pgh.pa.us The problem

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Christoph Berg
Re: Joe Conway 2019-03-29 > >> echo "\password :\"role\"" | psql -v role='my role' > > > > It is on-topic because the reason we can't just tell people to replace > > createuser $foo > > with > > psql -c "create user $foo" > > is because $foo might need escaping. > > > > IMHO if we find an wa

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Robert Haas
On Fri, Mar 29, 2019 at 3:28 PM Alvaro Herrera wrote: > On 2019-Mar-29, Robert Haas wrote: > > On Fri, Mar 29, 2019 at 2:16 PM Alvaro Herrera > > wrote: > > > I think we should consider a new column of an array type, where we could > > > put things like the list of PIDs to be waited for, the lis

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Joe Conway
On 3/29/19 3:43 PM, Christoph Berg wrote: > Re: Joe Conway 2019-03-29 <48e5efaf-7ea2-ed70-a803-949bbfec8...@joeconway.com> >> echo "\password :\"role\"" | psql -v role='my role' >> Enter new password: >> Enter it again: >> >> That said, this is kind of off the topic of this thread. > > It is on-t

Re: New vacuum option to do only freezing

2019-03-29 Thread Robert Haas
On Fri, Mar 29, 2019 at 12:27 PM Masahiko Sawada wrote: > Yeah, but since multiple relations might be specified in VACUUM > command we need to process index_cleanup option after opened each > relations. Maybe we need to process all options except for > INDEX_CLEANUP in ExecVacuum() and pass Vacuum

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Christoph Berg
Re: Joe Conway 2019-03-29 <48e5efaf-7ea2-ed70-a803-949bbfec8...@joeconway.com> > echo "\password :\"role\"" | psql -v role='my role' > Enter new password: > Enter it again: > > That said, this is kind of off the topic of this thread. It is on-topic because the reason we can't just tell people to

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Chapman Flack
On 3/29/19 3:32 PM, Joe Conway wrote: > pg_util > > Of course that does not lend itself to symlinking for backward > compatibility, does it? If there is a way I am not familiar with it. On Unix-like systems, you can have pg_util look at argv[0] to see if it was called createuser or what not.

Re: partitioned tables referenced by FKs

2019-03-29 Thread Jesper Pedersen
Hi Alvaro, On 3/28/19 2:59 PM, Alvaro Herrera wrote: I ended up revising the dependencies that we give to the constraint in the partition -- instead of giving it partition-type dependencies, we give it an INTERNAL dependency. Now when you request to drop the partition, it says this: create tab

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Christoph Berg
Re: Pavel Stehule 2019-03-29 > > Other idea: If we don't want to reinvent a new tool, how about > > supporting prepared statements in psql? > > > > psql -c 'create user %i' --args 'bob w. space' > > > > Prepared statements cannot be DDL commands. "Prepared" in the sense of what format() does.

Re: Enable data checksums by default

2019-03-29 Thread Christoph Berg
Re: Bernd Helmle 2019-03-29 <3586bb9345a59bfc8d13a50a7c729be1ee6759fd.ca...@oopsware.de> > Am Freitag, den 29.03.2019, 23:10 +0900 schrieb Michael Paquier: > > > > I can't really believe that many people set up shared_buffers at > > 128kB > > which would cause such a large number of page eviction

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Joe Conway
On 3/29/19 3:01 PM, Pavel Stehule wrote: > But psql has safe escaping via :"xxx" notation. So some like > > psql -c 'create role :"role"' -v role='my role' ... > > But what I know the psql variables are not evaluated for -c query You can do this: echo "create role :\"role\"" | psql -v role='my r

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Robert Haas wrote: > On Fri, Mar 29, 2019 at 2:16 PM Alvaro Herrera > wrote: > > I think we should consider a new column of an array type, where we could > > put things like the list of PIDs to be waited for, the list of OIDs of > > index to rebuild, or the list of partitions to

Re: Enable data checksums by default

2019-03-29 Thread Bernd Helmle
Am Freitag, den 29.03.2019, 23:10 +0900 schrieb Michael Paquier: > > I can't really believe that many people set up shared_buffers at > 128kB > which would cause such a large number of page evictions, but I can > believe that many users have shared_buffers set to its default value > and that we ar

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Robert Haas
On Fri, Mar 29, 2019 at 2:16 PM Alvaro Herrera wrote: > I think we should consider a new column of an array type, where we could > put things like the list of PIDs to be waited for, the list of OIDs of > index to rebuild, or the list of partitions to build the index on. This has to work with a fi

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-03-29 Thread Shaun Thomas
Another ping on this Incremental Sort patch. Alexander, you'd noted that you would try to get it into subsequent Commit Fests with improvements you've been considering, but I don't see it in anything but 2018-11. Have you abandoned this as a maintainer? If so, it would be nice to know so someone e

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Pavel Stehule
pá 29. 3. 2019 v 19:50 odesílatel Christoph Berg napsal: > Re: Tom Lane 2019-03-29 <19517.1553876...@sss.pgh.pa.us> > > >> Or perhaps better, allow pg_ctl to grow new subcommands for those > > >> tasks? > > > > > We'd need to be careful to somehow delineate commands that need access > > > to the

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Christoph Berg
Re: Tom Lane 2019-03-29 <19517.1553876...@sss.pgh.pa.us> > >> Or perhaps better, allow pg_ctl to grow new subcommands for those > >> tasks? > > > We'd need to be careful to somehow delineate commands that need access > > to the data directory / run locally on the server from the ones that > > just

Re: speeding up planning with partitions

2019-03-29 Thread Tom Lane
Amit Langote writes: > On 2019/03/29 7:38, Tom Lane wrote: >> 2. I seriously dislike what's been done in joinrels.c, too. That >> really seems like a kluge (and I haven't had time to study it >> closely). > Those hunks account for the fact that pruned partitions, for which we no > longer create

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Alvaro Herrera wrote: > So, CLUSTER and ALTER TABLE rewrites only do non-concurrent index > builds; and REINDEX can reuse pretty much the same wait-for metrics > columns as CIC. So I think it's okay if I move only the metrics that > conflict for index_build. The attached version

Re: partitioned tables referenced by FKs

2019-03-29 Thread Jesper Pedersen
Hi, On 3/29/19 12:29 PM, Alvaro Herrera wrote: On 2019-Mar-29, Jesper Pedersen wrote: Maybe the "(" / ")" in the CASCADE description should be removed from ref/drop_table.sgml as part of this patch. I'm not sure what text you propose to remove? Just the attached. Should catalogs.sgml be

Proposal: autovacuum_max_queue_depth

2019-03-29 Thread Chris Travers
Hi everyone. I would like to flesh this out in terms fo feedback before creating a patch. The Problem In large production systems often you can have problems when autovacuum is not tuned aggressively enough. This leads to long autovacuum runs when they happen, and autovacuum ends up eventually

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Tom Lane wrote: > Christoph Berg writes: > > What might possibly make sense is to add options to psql to > > facilitate common tasks: > > > psql --createdb foo > > psql --createuser bar --superuser > > psql --reindex foo > > That's a thought. Or perhaps better, allow pg_ctl to

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Andres Freund
On 2019-03-29 16:20:54 +, Simon Riggs wrote: > On Fri, 29 Mar 2019 at 16:12, Andres Freund wrote: > > > > On 2019-03-29 15:58:14 +, Simon Riggs wrote: > > > On Fri, 29 Mar 2019 at 15:29, Andres Freund wrote: > > > > That's far from a trivial feature imo. It seems quite possible that > >

Re: speeding up planning with partitions

2019-03-29 Thread Tom Lane
I wrote: > Amit Langote writes: >> About the XXX: I think resetting inh flag is unnecessary, so we should >> just remove the line. > Possibly. I hadn't had time to follow up the XXX annotation. Now I have ... Yeah, it seems we can just drop that and leave the flag alone. We'll end up running

Re: partitioned tables referenced by FKs

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Jesper Pedersen wrote: > Thanks ! > > Maybe the "(" / ")" in the CASCADE description should be removed from > ref/drop_table.sgml as part of this patch. I'm not sure what text you propose to remove? > Should catalogs.sgml be updated for this case ? I'm not adding any new depen

Re: New vacuum option to do only freezing

2019-03-29 Thread Masahiko Sawada
On Fri, Mar 29, 2019 at 10:46 PM Robert Haas wrote: > > On Fri, Mar 29, 2019 at 2:16 AM Masahiko Sawada wrote: > > Attached updated patches. These patches are applied on top of 0001 > > patch on parallel vacuum thread[1]. > > +bool index_cleanup = true; /* by default */ > > I think we should

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Tom Lane
Andres Freund writes: > On 2019-03-29 11:41:26 -0400, Tom Lane wrote: >> Or perhaps better, allow pg_ctl to grow new subcommands for those >> tasks? > We'd need to be careful to somehow delineate commands that need access > to the data directory / run locally on the server from the ones that > ju

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Peter Geoghegan
On Fri, Mar 29, 2019 at 9:12 AM Andres Freund wrote: > But even so, you can't have unlogged changes that you then rely on. Even > if there's no torn page issue. Currently BTP_HAS_GARBAGE and > ItemIdMarkDead() are treated as hints - if we want to guarantee all > these are accurate, I don't quite

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Simon Riggs
On Fri, 29 Mar 2019 at 16:12, Andres Freund wrote: > On 2019-03-29 15:58:14 +, Simon Riggs wrote: > > On Fri, 29 Mar 2019 at 15:29, Andres Freund wrote: > > > That's far from a trivial feature imo. It seems quite possible that > we'd > > > end up with increased overhead, because the current

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Andres Freund
Hi, On 2019-03-29 15:58:14 +, Simon Riggs wrote: > On Fri, 29 Mar 2019 at 15:29, Andres Freund wrote: > > That's far from a trivial feature imo. It seems quite possible that we'd > > end up with increased overhead, because the current logic can get away > > with only doing hint bit style writ

Re: Pluggable Storage - Andres's take

2019-03-29 Thread Andres Freund
On 2019-03-29 18:38:46 +1100, Haribabu Kommi wrote: > As I see that your are fixing some typos of the code that is committed, > I just want to share some more corrections that I found in the patches > that are committed till now. Pushed both, thanks!

Re: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Justin Pryzby
On Fri, Mar 29, 2019 at 03:53:05PM +, Bossart, Nathan wrote: > I noticed a very small typo in the documentation for this feature. I submit a bunch more changes for consideration, attached. >From dafdb15fb3e7c69de82a2206c9bf07588b5665ce Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Fri, 2

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Simon Riggs
On Fri, 29 Mar 2019 at 15:29, Andres Freund wrote: > On 2019-03-29 09:37:11 +, Simon Riggs wrote: > > > While trying to understand this, I see there is an even better way to > > optimize this. Since we are removing dead index tuples, we could alter > the > > killed index tuple interface so

Re: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Bossart, Nathan
I noticed a very small typo in the documentation for this feature. diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index ccabb330cb..e45bf86c8d 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -349,7 +349,7 @@ REINDEX [ ( VERBOSE ) ] { IN

Re: Online verification of checksums

2019-03-29 Thread Magnus Hagander
On Fri, Mar 29, 2019 at 4:30 PM Stephen Frost wrote: > Greetings, > > * Magnus Hagander (mag...@hagander.net) wrote: > > On Thu, Mar 28, 2019 at 10:19 PM Tomas Vondra < > tomas.von...@2ndquadrant.com> > > wrote: > > > > > On Thu, Mar 28, 2019 at 01:11:40PM -0700, Andres Freund wrote: > > > >Hi, >

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Andres Freund
Hi, On 2019-03-29 11:41:26 -0400, Tom Lane wrote: > Or perhaps better, allow pg_ctl to grow new subcommands for those > tasks? We'd need to be careful to somehow delineate commands that need access to the data directory / run locally on the server from the ones that just needs a client connection

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Joe Conway
On 3/29/19 11:44 AM, Daniel Gustafsson wrote: > On Friday, March 29, 2019 4:41 PM, Tom Lane wrote: > >> Christoph Berg m...@debian.org writes: >> >> > What might possibly make sense is to add options to psql to >> > facilitate common tasks: >> >> > psql --createdb foo >> > psql --createuser bar -

Re: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Andres Freund
Hi, On 2019-03-29 11:47:10 -0400, Robert Treat wrote: > On Fri, Mar 29, 2019 at 3:28 AM Peter Eisentraut > wrote: > > > > On 2019-03-28 09:07, Sergei Kornilov wrote: > > > Unfortunately patch does not apply due recent commits. Any chance this > > > can be fixed (and even committed in pg12)? > >

Re: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Robert Treat
On Fri, Mar 29, 2019 at 3:28 AM Peter Eisentraut wrote: > > On 2019-03-28 09:07, Sergei Kornilov wrote: > > Unfortunately patch does not apply due recent commits. Any chance this can > > be fixed (and even committed in pg12)? > > Committed :) > Given this has been committed I've probably missed

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Daniel Gustafsson
On Friday, March 29, 2019 4:41 PM, Tom Lane wrote: > Christoph Berg m...@debian.org writes: > > > What might possibly make sense is to add options to psql to > > facilitate common tasks: > > > psql --createdb foo > > psql --createuser bar --superuser > > psql --reindex foo > > That's a thought. O

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Tom Lane
Christoph Berg writes: > What might possibly make sense is to add options to psql to > facilitate common tasks: > psql --createdb foo > psql --createuser bar --superuser > psql --reindex foo That's a thought. Or perhaps better, allow pg_ctl to grow new subcommands for those tasks?

Re: Online verification of checksums

2019-03-29 Thread Andres Freund
Hi, On 2019-03-29 11:38:02 -0400, Stephen Frost wrote: > The server-side function would essentially lock the page against i/o, > re-read it off disk into an independent location, unlock the page, then > calculate the checksum and report back? Right. I think there's a few minor variations of how t

Re: Online verification of checksums

2019-03-29 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-03-29 11:30:15 -0400, Stephen Frost wrote: > > * Magnus Hagander (mag...@hagander.net) wrote: > > > On Thu, Mar 28, 2019 at 10:19 PM Tomas Vondra > > > > > > wrote: > > > > On Thu, Mar 28, 2019 at 01:11:40PM -0700, Andres Freund wr

Re: table_privileges view always show object owner as a grantor

2019-03-29 Thread Ashutosh Sharma
On Fri, Mar 29, 2019 at 8:15 PM Tom Lane wrote: > > Laurenz Albe writes: > > Ashutosh Sharma wrote: > >> I noticed that irrespective of whoever grants privileges on an object, > >> it's always the object owner who is seen as a grantor in the output of > >> table_privileges view. > > The above is

Re: Online verification of checksums

2019-03-29 Thread Andres Freund
Hi, On 2019-03-29 11:30:15 -0400, Stephen Frost wrote: > * Magnus Hagander (mag...@hagander.net) wrote: > > On Thu, Mar 28, 2019 at 10:19 PM Tomas Vondra > > wrote: > > > On Thu, Mar 28, 2019 at 01:11:40PM -0700, Andres Freund wrote: > > > >Hi, > > > > > > > >On 2019-03-28 21:09:22 +0100, Michael

Re: partitioned tables referenced by FKs

2019-03-29 Thread Jesper Pedersen
Hi, On 3/29/19 11:22 AM, Alvaro Herrera wrote: On 2019-Mar-29, Jesper Pedersen wrote: Could expand a bit on the change to DEPENDENCY_INTERNAL instead of DEPENDENCY_PARTITION_PRI / DEPENDENCY_PARTITION_SEC ? The PARTITION dependencies work in a way that doesn't do what we want. Admittedly, ne

Re: Online verification of checksums

2019-03-29 Thread Stephen Frost
Greetings, * Magnus Hagander (mag...@hagander.net) wrote: > On Thu, Mar 28, 2019 at 10:19 PM Tomas Vondra > wrote: > > > On Thu, Mar 28, 2019 at 01:11:40PM -0700, Andres Freund wrote: > > >Hi, > > > > > >On 2019-03-28 21:09:22 +0100, Michael Banck wrote: > > >> I agree that the current patch mig

Re: speeding up planning with partitions

2019-03-29 Thread Tom Lane
Amit Langote writes: > Here are some comments on v38. Thanks for looking it over! I'll just reply to points worth discussing: > -Assert(rte->rtekind == RTE_RELATION || > - rte->rtekind == RTE_SUBQUERY); > -add_appendrel_other_rels(root, rel, rti); > +

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Andres Freund
Hi, On 2019-03-29 09:37:11 +, Simon Riggs wrote: > This commit message was quite confusing. It took me a while to realize this > relates to btree index deletes and that what you mean is that we are > calculcating the latestRemovedXid for index entries. That is related to but > not same thing a

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Christoph Berg
Re: Tatsuo Ishii 2019-03-29 <20190329.100407.1159785913847835944.t-is...@sraoss.co.jp> > If we were to invent new command names, what about doing similar to > git? I mean something like: > > pgsql createdb That is pretty close to "psql" and it will be utterly confusing for new users. And ev

Re: partitioned tables referenced by FKs

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Jesper Pedersen wrote: > Could expand a bit on the change to DEPENDENCY_INTERNAL instead of > DEPENDENCY_PARTITION_PRI / DEPENDENCY_PARTITION_SEC ? The PARTITION dependencies work in a way that doesn't do what we want. Admittedly, neither does INTERNAL, but at least it's less bad.

Re: partitioned tables referenced by FKs

2019-03-29 Thread Jesper Pedersen
Hi Alvaro, On 3/28/19 2:59 PM, Alvaro Herrera wrote: I ended up revising the dependencies that we give to the constraint in the partition -- instead of giving it partition-type dependencies, we give it an INTERNAL dependency. Now when you request to drop the partition, it says this: create tab

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Andres Freund wrote: > Hi, > > On 2019-03-29 12:02:18 -0300, Alvaro Herrera wrote: > > I just noticed that the CLUSTER calls index_build, which my patch > > modifies to include additional progress metrics; this means that during > > the index build phase, the metrics set by CLUSTE

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Alvaro Herrera wrote: > I just noticed that the CLUSTER calls index_build, which my patch > modifies to include additional progress metrics; this means that during > the index build phase, the metrics set by CLUSTER will be trashed by the > ones my patch introduces. Indeed: pid

RE: REINDEX CONCURRENTLY 2.0

2019-03-29 Thread Shinoda, Noriyoshi (PN Japan A&PS Delivery)
Hi hackers, I tried this great feature for partition index. The first time the REINDEX TABLE CONCURRENTLY statement is executed to the partition, then an error occurs. The second run succeeds but leaves an index with an INVALID status. I think this is not the desired behaviour. # TEST postgres

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Andres Freund
Hi, On 2019-03-29 12:02:18 -0300, Alvaro Herrera wrote: > I just noticed that the CLUSTER calls index_build, which my patch > modifies to include additional progress metrics; this means that during > the index build phase, the metrics set by CLUSTER will be trashed by the > ones my patch introduce

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-29 Thread Alvaro Herrera
I just noticed that the CLUSTER calls index_build, which my patch modifies to include additional progress metrics; this means that during the index build phase, the metrics set by CLUSTER will be trashed by the ones my patch introduces. -- Álvaro Herrerahttps://www.2ndQuadrant.com

Re: log bind parameter values on error

2019-03-29 Thread Alexey Bashtanov
Hello and sorry for weeks of silence. Hello Anders and Peter, Thanks for your messages. Please see the new patch version attached. In my testing, I couldn't get this patch to do anything. Could you please share your testing steps? Sure. Provided you're in the postgres checkout and you've ru

Re: table_privileges view always show object owner as a grantor

2019-03-29 Thread Tom Lane
Laurenz Albe writes: > Ashutosh Sharma wrote: >> I noticed that irrespective of whoever grants privileges on an object, >> it's always the object owner who is seen as a grantor in the output of >> table_privileges view. The above is demonstrably false ... regression=# create user alice; CREATE R

clean up pg_checksums.sgml

2019-03-29 Thread Justin Pryzby
PFA patch with minor improvements to documentation. Also, what do you think about changing user-facing language from "check checksum" to "verify checksum" ? I see that commit ed308d78 actually moved in the other direction, but I preferred "verify". >From d8e627cf340e5384d59ab4fc3f3d0b4891a5b1c0 M

Re: PostgreSQL pollutes the file system

2019-03-29 Thread Fred .Flintstone
I think that would be amazing! It would be great! On Fri, Mar 29, 2019 at 4:01 AM Tatsuo Ishii wrote: > > > Andreas Karlsson writes: > >> On 3/27/19 3:26 PM, Tomas Vondra wrote: > >>> That is true, of course. But are there actual examples of such conflicts > >>> in practice? I mean, are there to

Re: partitioned tables referenced by FKs

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Robert Haas wrote: > On Wed, Mar 20, 2019 at 11:58 AM Alvaro Herrera > wrote: > > constraint is dropped. I can only think of ugly data structure to > > support this, and adding enough hooks in dependency.c to support this is > > going to be badly received. > > I don't know why d

unsuscribe

2019-03-29 Thread Enrique Kurth Schoenfeld Escobar

Re: [PG 9.6]make is failing for test_decoding contrib module.

2019-03-29 Thread tushar
On 03/29/2019 06:12 PM, Robert Haas wrote: Kashif Jeeshan? Ohh, Please read - Kashif Zeeshan.  Sorry for the typo. -- regards,tushar EnterpriseDB https://www.enterprisedb.com/ The Enterprise PostgreSQL Company

Re: pg_ctl on windows can't open postmaster.pid: Permission denied

2019-03-29 Thread Michael Paquier
On Sat, Mar 30, 2019 at 12:13:23AM +1300, Thomas Munro wrote: > This is probably a stupid question, but after commit 0ba06e0, > shouldn't pg_ctl.c have fopen defined as pgwin32_fopen by port.h, > because it was included by c.h, because it was included by > postgres_fe.h? Yeah, normally pg_ctl shou

Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru

2019-03-29 Thread Alvaro Herrera
On 2019-Mar-29, Michael Paquier wrote: > On Fri, Mar 29, 2019 at 09:11:47AM -0400, Andrew Dunstan wrote: > > +                (errmsg_internal("found vacuum to prevent wraparound of > > table \"%s.%s.%s\" to be not aggressive, so skipping", > > > > This might convey something to hackers, but I do

Re: table_privileges view always show object owner as a grantor

2019-03-29 Thread Laurenz Albe
Ashutosh Sharma wrote: > I noticed that irrespective of whoever grants privileges on an object, > it's always the object owner who is seen as a grantor in the output of > table_privileges view. > Isn't that a wrong information ? If > incase that isn't wrong then may i k

Re: Enable data checksums by default

2019-03-29 Thread Michael Paquier
On Fri, Mar 29, 2019 at 11:16:11AM +0100, Bernd Helmle wrote: > So between ~7% to 18% impact with checksums in this specific case here. I can't really believe that many people set up shared_buffers at 128kB which would cause such a large number of page evictions, but I can believe that many users

Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru

2019-03-29 Thread Michael Paquier
On Fri, Mar 29, 2019 at 09:11:47AM -0400, Andrew Dunstan wrote: > +                (errmsg_internal("found vacuum to prevent wraparound of > table \"%s.%s.%s\" to be not aggressive, so skipping", > > This might convey something to hackers, but I doubt it will convey much > to regular users. Perhap

Re: [GSoC 2019] Proposal: Develop Performance Farm Database and Website

2019-03-29 Thread Ilaria
Hello, Thanks for the answer. This project is on the official PostgreSQL project list of GSoC 2019, and potential mentors are stated there. I trust mentors’ judgement on outlining the work and the tasks to be done in three months, and there is the previous student’s work to use as example if

Re: Offline enabling/disabling of data checksums

2019-03-29 Thread Michael Paquier
On Tue, Mar 26, 2019 at 01:41:38PM +0100, Fabien COELHO wrote: >> I am not sure that "checksum status" is a correct term. It seems to >> me that "same configuration for data checksums as before the tool ran" >> or something like that would be more correct. > > Possibly, I cannot say. I have put

Re: partitioned tables referenced by FKs

2019-03-29 Thread Robert Haas
On Thu, Mar 28, 2019 at 2:59 PM Alvaro Herrera wrote: > I ended up revising the dependencies that we give to the constraint in > the partition -- instead of giving it partition-type dependencies, we > give it an INTERNAL dependency. Now when you request to drop the > partition, it says this: > >

Re: partitioned tables referenced by FKs

2019-03-29 Thread Robert Haas
On Wed, Mar 20, 2019 at 11:58 AM Alvaro Herrera wrote: > constraint is dropped. I can only think of ugly data structure to > support this, and adding enough hooks in dependency.c to support this is > going to be badly received. I don't know why dependency.c doesn't handle this internally. If I

Re: New vacuum option to do only freezing

2019-03-29 Thread Robert Haas
On Fri, Mar 29, 2019 at 2:16 AM Masahiko Sawada wrote: > Attached updated patches. These patches are applied on top of 0001 > patch on parallel vacuum thread[1]. +bool index_cleanup = true; /* by default */ I think we should instead initialize index_cleanup to the reloption value, if there

Re: [PG 9.6]make is failing for test_decoding contrib module.

2019-03-29 Thread tushar
On 03/29/2019 06:12 PM, Robert Haas wrote: On Fri, Mar 29, 2019 at 8:24 AM tushar wrote: Found by one of the my colleague - Kashif Jeeshan , in PG 9.6 - make is Kashif Jeeshan? :-) , actually he is also working  on logical replication on standbys testing - whenever he has some bandwidth (On/

Re: [HACKERS] Block level parallel vacuum

2019-03-29 Thread Masahiko Sawada
On Fri, Mar 29, 2019 at 9:28 PM Robert Haas wrote: > > On Thu, Mar 28, 2019 at 10:27 PM Masahiko Sawada > wrote: > > You're right, the previous patches are wrong. Attached the updated > > version patches. > > 0001 looks good now. Committed. > Thank you! Regards, -- Masahiko Sawada NIPPON TEL

Re: jsonpath

2019-03-29 Thread Alexander Korotkov
On Thu, Mar 28, 2019 at 7:43 PM Andrew Dunstan wrote: > On 3/28/19 9:50 AM, Tom Lane wrote: > > Andres Freund writes: > >> On March 28, 2019 9:31:14 AM EDT, Tom Lane wrote: > >>> Has anybody gotten through a valgrind run on this code yet? > >> Skink has successfully passed since - but that's x86

Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru

2019-03-29 Thread Andrew Dunstan
On 3/29/19 7:51 AM, Michael Paquier wrote: > On Sat, Mar 09, 2019 at 10:15:37AM +0900, Michael Paquier wrote: >> I am adding an open item about that. I think I could commit the >> patch, but I need to study it a bit more first. > So, coming back to this thread, and studying the problem again, it

Re: fsync error handling in pg_receivewal, pg_recvlogical

2019-03-29 Thread Michael Paquier
On Fri, Mar 29, 2019 at 12:48:09PM +0100, Peter Eisentraut wrote: > Do we need to review the fsync error handling in pg_receivewal and > pg_recvlogical, following the recent backend changes? The current > default behavior is that these tools will log fsync errors and then > reconnect and proceed w

Re: Add exclusive backup deprecation notes to documentation

2019-03-29 Thread Peter Eisentraut
On 2019-03-29 13:54, Magnus Hagander wrote: > Is the changes to the messages going to cause issues or weirdness for > translators? That would be a reason not to backpatch it. Without that, > I'm leaning towards backpatching it. Note that the messages refer to recovery.signal, so a backpatch would

Re: Add exclusive backup deprecation notes to documentation

2019-03-29 Thread Magnus Hagander
On Fri, Mar 29, 2019 at 1:49 PM David Steele wrote: > On 3/29/19 12:46 PM, Robert Haas wrote: > > On Fri, Mar 29, 2019 at 8:45 AM David Steele > wrote: > >> Are we planning to back-patch this? The deprecation was added to the > >> docs in 9.6 -- I think these clarifications would be helpful. >

Re: [GSoC 2019] Proposal: Develop Performance Farm Database and Website

2019-03-29 Thread Peter Eisentraut
On 2019-03-29 13:04, Robert Haas wrote: > On Tue, Mar 26, 2019 at 9:10 AM Ila B. wrote: >> I am Ilaria Battiston, an aspiring GSoC student, and I would love to have a >> feedback on the first draft of my Google Summer of Code proposal. The >> project is "Develop Performance Farm Database and Web

  1   2   >