RE: speeding up planning with partitions

2018-09-03 Thread Kato, Sho
Hi, Amit. Thank you for your reply. > I didn't load any data into tables when running the tests, because these > patches are meant for reducing planner latency. More specifically, they're > addressed to fix the current planner behavior that its latency increases with > increasing number of

Re: pg_verify_checksums failure with hash indexes

2018-09-03 Thread Dilip Kumar
On Tue, Sep 4, 2018 at 10:14 AM, Amit Kapila wrote: > On Mon, Sep 3, 2018 at 2:44 PM Dilip Kumar wrote: >> On Mon, Sep 3, 2018 at 8:37 AM, Amit Kapila wrote: >> > On Sat, Sep 1, 2018 at 10:28 AM Dilip Kumar wrote: >> >> >> >> I think if we compute with below formula which I suggested upthread

Re: Accidental removal of a file causing various problems

2018-09-03 Thread Pavan Deolasee
On Sat, Aug 25, 2018 at 1:15 AM Tom Lane wrote: > Actually, I think the main point is given that we've somehow got into > a situation like that, how do we get out again? > I and Alvaro discussed this off-list a bit and we came up with couple of ideas. 1. Reserve some buffers in the shared

Re: pg_verify_checksums failure with hash indexes

2018-09-03 Thread Amit Kapila
On Mon, Sep 3, 2018 at 2:44 PM Dilip Kumar wrote: > On Mon, Sep 3, 2018 at 8:37 AM, Amit Kapila wrote: > > On Sat, Sep 1, 2018 at 10:28 AM Dilip Kumar wrote: > >> > >> I think if we compute with below formula which I suggested upthread > >> > >> #define HASH_MAX_BITMAPS

Re: pointless check in RelationBuildPartitionDesc

2018-09-03 Thread Alvaro Herrera
On 2018-Sep-04, Amit Langote wrote: > On 2018/09/04 10:19, Michael Paquier wrote: > > On Tue, Sep 04, 2018 at 09:47:07AM +0900, Amit Langote wrote: > >> On 2018/09/04 6:39, Alvaro Herrera wrote: > >>> After looking, it seems that this is just self-inflicted pain: for some > >>> reason, we store

Re: pg_verify_checksums failure with hash indexes

2018-09-03 Thread Amit Kapila
On Mon, Sep 3, 2018 at 7:21 PM Tom Lane wrote: > > Amit Kapila writes: > > AFAICS, this problem exists in 9.6 and prior branches as well, > > although, I can't test it. I am not sure whether we need to backpatch > > this beyond 10 (where hash indexes are WAL logged) as prior to that > >

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

2018-09-03 Thread Noah Misch
On Sun, Feb 12, 2017 at 09:32:36AM -0800, Tom Lane wrote: > It's not so much poor choices as the cost of the optimization attempt --- > if there's a K-relation OR clause, this will increase the cost of planning > by a factor approaching K+1, whether or not you get a better plan out of > it. I ran

Re: pointless check in RelationBuildPartitionDesc

2018-09-03 Thread Amit Langote
On 2018/09/04 10:19, Michael Paquier wrote: > On Tue, Sep 04, 2018 at 09:47:07AM +0900, Amit Langote wrote: >> On 2018/09/04 6:39, Alvaro Herrera wrote: >>> After looking, it seems that this is just self-inflicted pain: for some >>> reason, we store the pg_inherits row for a partition, and

Re: pointless check in RelationBuildPartitionDesc

2018-09-03 Thread Michael Paquier
On Tue, Sep 04, 2018 at 09:47:07AM +0900, Amit Langote wrote: > On 2018/09/04 6:39, Alvaro Herrera wrote: >> After looking, it seems that this is just self-inflicted pain: for some >> reason, we store the pg_inherits row for a partition, and immediately >> afterwards compute and store its

Re: libpq debug log

2018-09-03 Thread Tom Lane
"Iwata, Aya" writes: > The purpose of this log acquisition I thought is to identify where is the > problem: > server side, application side or traffic. TBH, I think the sort of logging you're proposing would be expensive enough that *it* would be the bottleneck in a lot of cases. A lot of

Re: Caching query plan costs

2018-09-03 Thread Bruce Momjian
On Mon, Sep 3, 2018 at 04:13:40PM -0700, Andres Freund wrote: > On September 3, 2018 3:01:29 PM PDT, Bruce Momjian > wrote: > >On Mon, Sep 3, 2018 at 02:53:59PM -0700, Andres Freund wrote: > >> On 2018-09-03 14:56:28 -0400, Bruce Momjian wrote: > >> > On Mon, Sep 3, 2018 at 11:42:31AM -0700,

Re: Slotification of partition tuple conversion

2018-09-03 Thread Amit Langote
Hi Amit, Thanks for the updated patch and sorry I couldn't reply sooner. On 2018/08/21 16:18, Amit Khandekar wrote: > On 21 August 2018 at 08:12, Amit Langote > wrote: >> Here are some comments on the patch: > > Thanks for the review. > >> >> +ConvertTupleSlot >> >> Might be a good idea to

Re: pointless check in RelationBuildPartitionDesc

2018-09-03 Thread Amit Langote
On 2018/09/04 6:39, Alvaro Herrera wrote: > I noticed this strange hack in RelationBuildPartitionDesc: > >/* > * It is possible that the pg_class tuple of a partition has not been > * updated yet to set its relpartbound field. The only case where > * this happens

Re: [HACKERS] Restricting maximum keep segments by repslots

2018-09-03 Thread Masahiko Sawada
Thank you for updating the patch. On Tue, Jul 31, 2018 at 6:11 PM, Kyotaro HORIGUCHI wrote: > Hello. > > At Tue, 24 Jul 2018 16:47:41 +0900, Masahiko Sawada > wrote in >> On Mon, Jul 23, 2018 at 4:16 PM, Kyotaro HORIGUCHI >> wrote: >> > Hello. >> > >> > At Fri, 20 Jul 2018 10:13:58 +0900,

Re: CREATE ROUTINE MAPPING

2018-09-03 Thread Masahiko Sawada
On Tue, Sep 4, 2018 at 5:48 AM, David Fetter wrote: > On Fri, Aug 31, 2018 at 05:18:26PM +0900, Masahiko Sawada wrote: >> On Thu, Jan 25, 2018 at 2:13 PM, David Fetter wrote: >> > On Thu, Jan 18, 2018 at 04:09:13PM -0500, Corey Huinker wrote: >> >> > >> >> > >> >> > > >> >> > > But other

Re: Pluggable Storage - Andres's take

2018-09-03 Thread Andres Freund
Hi, Thanks for the patches! On 2018-09-03 19:06:27 +1000, Haribabu Kommi wrote: > I found couple of places where the zheap is using some extra logic in > verifying > whether it is zheap AM or not, based on that it used to took some extra > decisions. > I am analyzing all the extra code that is

RE: libpq debug log

2018-09-03 Thread Iwata, Aya
> > I'm going to propose libpq debug log for analysis of queries on the > > application > side. > > I think that it is useful to determine whether the cause is on the > > application > side or the server side when a slow query occurs. > > Do you mean you want to monitor the protocol message

Re: Loaded footgun open_datasync on Windows

2018-09-03 Thread Michael Paquier
On Sat, Sep 01, 2018 at 05:07:21PM -0700, Noah Misch wrote: > If you grep src/bin/pg_upgrade for WIN32, roughly a third of the hits are > workarounds for this problem. I agree with Michael that removing those > workarounds would be a good test of frontend pgwin32_open() and worth > including in

Re: Add SKIP LOCKED to VACUUM and ANALYZE

2018-09-03 Thread Michael Paquier
On Sun, Jul 29, 2018 at 10:56:24PM +, Bossart, Nathan wrote: > Since vacuum_rel() already obtains an AccessExclusiveLock on the > relation for VACUUM FULL, we might be able to skip altering > cluster_rel(). I think we will need to alter it if we are going to > add SKIP LOCKED to CLUSTER,

Re: Caching query plan costs

2018-09-03 Thread Andres Freund
On September 3, 2018 3:01:29 PM PDT, Bruce Momjian wrote: >On Mon, Sep 3, 2018 at 02:53:59PM -0700, Andres Freund wrote: >> On 2018-09-03 14:56:28 -0400, Bruce Momjian wrote: >> > On Mon, Sep 3, 2018 at 11:42:31AM -0700, Andres Freund wrote: >> > > > and JIT, so it doesn't have to be 100%

Re: Issues while building PG in MS Windows, using MSYS2 and MinGW-w64

2018-09-03 Thread Andre_Mikulec
Reprinted with corrections. I did everything in "MINGW64" not "MSYS" Hi, I had this problem. The solution is to turn off AVG anti-virus. In the past, I could both compile PostgreSQL and keep 'AVG on' at the same time. The case * does not seem that way anymore * With AVG 'antivirus on' the

Re: Issues while building PG in MS Windows, using MSYS2 and MinGW-w64

2018-09-03 Thread Andre_Mikulec
Hi, I had this problem. The solution is to turn off AVG anti-virus. In the past, I could both compile PostgreSQL and keep 'AVG on' at the same time. The case * does not seem that way anymore * With AVG 'antivirus on' the following occurs. AnonymousUser@ANONYMOUST MSYS /w/R-3.5._/postgres/build

Re: Caching query plan costs

2018-09-03 Thread Bruce Momjian
On Mon, Sep 3, 2018 at 02:53:59PM -0700, Andres Freund wrote: > On 2018-09-03 14:56:28 -0400, Bruce Momjian wrote: > > On Mon, Sep 3, 2018 at 11:42:31AM -0700, Andres Freund wrote: > > > > and JIT, so it doesn't have to be 100% accurate. > > > > > > JIT decision is done after main planning, so

Re: Caching query plan costs

2018-09-03 Thread Andres Freund
On 2018-09-03 14:56:28 -0400, Bruce Momjian wrote: > On Mon, Sep 3, 2018 at 11:42:31AM -0700, Andres Freund wrote: > > > and JIT, so it doesn't have to be 100% accurate. > > > > JIT decision is done after main planning, so we know the cost. > > Well, as I remember, we are considering disabling

pointless check in RelationBuildPartitionDesc

2018-09-03 Thread Alvaro Herrera
I noticed this strange hack in RelationBuildPartitionDesc: /* * It is possible that the pg_class tuple of a partition has not been * updated yet to set its relpartbound field. The only case where * this happens is when we open the parent relation to check using its

Re: [HACKERS] Bug in to_timestamp().

2018-09-03 Thread Alexander Korotkov
Hi! Sorry for very long reply. On Thu, Aug 16, 2018 at 11:44 PM David G. Johnston wrote: > If the new behavior is an error I don't really have a problem since the need > to fix one's queries will be obvious. > > "So length of last group of spaces/separators in the pattern should be > greater

Re: CREATE ROUTINE MAPPING

2018-09-03 Thread David Fetter
On Fri, Aug 31, 2018 at 05:18:26PM +0900, Masahiko Sawada wrote: > On Thu, Jan 25, 2018 at 2:13 PM, David Fetter wrote: > > On Thu, Jan 18, 2018 at 04:09:13PM -0500, Corey Huinker wrote: > >> > > >> > > >> > > > >> > > But other situations seem un-handle-able to me: > >> > > > >> > > SELECT

Re: Caching query plan costs

2018-09-03 Thread Tomas Vondra
On 09/03/2018 08:56 PM, Bruce Momjian wrote: > On Mon, Sep 3, 2018 at 11:42:31AM -0700, Andres Freund wrote: >> On September 3, 2018 11:33:35 AM PDT, Bruce Momjian >> wrote: >>> On Mon, Sep 3, 2018 at 01:30:33PM -0400, Tom Lane wrote: Bruce Momjian writes: > What if we globally or

Re: Online verification of checksums

2018-09-03 Thread Tomas Vondra
Hi, The patch is mostly copying the verification / retry logic from basebackup.c, but I think it omitted a rather important detail that makes it incorrect in the presence of concurrent writes. The very first thing basebackup does is this: startptr = do_pg_start_backup(...); i.e. it waits

Re: A strange GiST error message or fillfactor of GiST build

2018-09-03 Thread Alexander Korotkov
On Mon, Sep 3, 2018 at 9:09 PM Andrey Borodin wrote: > > 3 сент. 2018 г., в 20:16, Alexander Korotkov > > написал(а): > > That's a good idea. Especially if we take into account that > > fillfactor is not a forever bad idea for GIST, it just doesn't look > > reasonable for current building

Re: Caching query plan costs

2018-09-03 Thread Bruce Momjian
On Mon, Sep 3, 2018 at 11:42:31AM -0700, Andres Freund wrote: > On September 3, 2018 11:33:35 AM PDT, Bruce Momjian > wrote: > >On Mon, Sep 3, 2018 at 01:30:33PM -0400, Tom Lane wrote: > >> Bruce Momjian writes: > >> > What if we globally or locally cache the _cost_ of plans, so we > >> > can

Re: Out arguments name of "pg_identify_object_as_address" function in 9.5.14 and 11beta3

2018-09-03 Thread Alvaro Herrera
On 2018-Sep-03, Tom Lane wrote: > Jean-Pierre Pelletier writes: > > Two of the out arguments name of function > > "pg_identify_object_as_address" are not as documented. > > Documentation says "name" and "args", but function returns > > "object_names" and "object_args". > > Hm, yeah. The

Re: Out arguments name of "pg_identify_object_as_address" function in 9.5.14 and 11beta3

2018-09-03 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> I do not think we can change the names of the output arguments; > Tom> it'd break existing queries. However, renaming input arguments > Tom> shouldn't affect anything. > What about callers that might be using named-argument notation?

Re: Out arguments name of "pg_identify_object_as_address" function in 9.5.14 and 11beta3

2018-09-03 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> I do not think we can change the names of the output arguments; Tom> it'd break existing queries. However, renaming input arguments Tom> shouldn't affect anything. What about callers that might be using named-argument notation? -- Andrew

Re: Caching query plan costs

2018-09-03 Thread Andres Freund
On September 3, 2018 11:33:35 AM PDT, Bruce Momjian wrote: >On Mon, Sep 3, 2018 at 01:30:33PM -0400, Tom Lane wrote: >> Bruce Momjian writes: >> > What if we globally or locally cache the _cost_ of plans, so we can >> > consult that cache before planning and enable certain >optimizations? >>

Re: Out arguments name of "pg_identify_object_as_address" function in 9.5.14 and 11beta3

2018-09-03 Thread Tom Lane
Jean-Pierre Pelletier writes: > Two of the out arguments name of function > "pg_identify_object_as_address" are not as documented. > Documentation says "name" and "args", but function returns > "object_names" and "object_args". Hm, yeah. The documentation of pg_get_object_address() seems

Re: Caching query plan costs

2018-09-03 Thread Bruce Momjian
On Mon, Sep 3, 2018 at 01:30:33PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > What if we globally or locally cache the _cost_ of plans, so we can > > consult that cache before planning and enable certain optimizations? > > But what would you use as cache key? And how's this help if we

Out arguments name of "pg_identify_object_as_address" function in 9.5.14 and 11beta3

2018-09-03 Thread Jean-Pierre Pelletier
Two of the out arguments name of function "pg_identify_object_as_address" are not as documented. Documentation says "name" and "args", but function returns "object_names" and "object_args". This query shows what the function returns: select (pg_identify_object_as_address(classId, ObjId,

Re: Hint to set owner for tablespace directory

2018-09-03 Thread avatar28
Thanks, that should be helpful. - Currently Junior Marketing Specialist @ CrazyCall -- Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html

Re: A strange GiST error message or fillfactor of GiST build

2018-09-03 Thread Andrey Borodin
> 3 сент. 2018 г., в 20:16, Alexander Korotkov > написал(а): > > That's a good idea. Especially if we take into account that > fillfactor is not a forever bad idea for GIST, it just doesn't look > reasonable for current building algorithm. So, we can decide to > ignore it, but if we would

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-09-03 Thread Fabien COELHO
Hello Yugo-san, I attached the rebased patch. Patch applies cleanly, compiles, "make check" is okay, although there are no specific test for the feature. Indeed, after investigation there is not a SINGLE test for the command:-( I think that some minimal tap-testing should be done. It

Re: Caching query plan costs

2018-09-03 Thread Tom Lane
Bruce Momjian writes: > What if we globally or locally cache the _cost_ of plans, so we can > consult that cache before planning and enable certain optimizations? But what would you use as cache key? And how's this help if we haven't seen a similar query before in the session?

Caching query plan costs

2018-09-03 Thread Bruce Momjian
This email thread from last year: https://www.postgresql.org/message-id/flat/CAMjNa7cC4X9YR-vAJS-jSYCajhRDvJQnN7m2sLH1wLh-_Z2bsw%40mail.gmail.com got stuck on when to check for certain optimizations in the planner. It was discussed that knowing a plan was going to be expensive _before_

Re: A strange GiST error message or fillfactor of GiST build

2018-09-03 Thread Alexander Korotkov
On Sat, Sep 1, 2018 at 9:45 PM Tom Lane wrote: > Alexander Korotkov writes: > > Thus, I would vote for removing GiST fillfactor altogether. Assuming > > we can't do this for compatibility reasons, I would vote for setting > > default GiST fillfactor to 100, and don't introduce new places where

Re: Progress reporting for pg_verify_checksums

2018-09-03 Thread Michael Banck
Hi, On Mon, Sep 03, 2018 at 11:21:32AM -0300, Alvaro Herrera wrote: > On 2018-Sep-01, Fabien COELHO wrote: > > > If -P was forgotten and pg_verify_checksums operates on a large cluster, > > > the caller can send SIGUSR1 to pg_verify_checksums to turn progress > > > status reporting on during

Re: Progress reporting for pg_verify_checksums

2018-09-03 Thread Alvaro Herrera
On 2018-Sep-01, Fabien COELHO wrote: > > If -P was forgotten and pg_verify_checksums operates on a large cluster, > > the caller can send SIGUSR1 to pg_verify_checksums to turn progress > > status reporting on during runtime. > > Hmmm. I cannot say I like the signal feature much. Would it make

Re: TR: pgadmin not displaying data from postgresql_fdw

2018-09-03 Thread Alvaro Herrera
On 2018-Aug-31, Olivier Leprêtre wrote: > Hi, > > > > Please find a question that didn't get an answer in the pgsql-sql list. I > hope I'll get an answer here. This seems a question for pgadmin-users instead: > So everything seems fine, except the pgadmin tree which does not display any >

Re: pg_verify_checksums failure with hash indexes

2018-09-03 Thread Tom Lane
Amit Kapila writes: > AFAICS, this problem exists in 9.6 and prior branches as well, > although, I can't test it. I am not sure whether we need to backpatch > this beyond 10 (where hash indexes are WAL logged) as prior to that > hash-indexes are anyway not-reliable. What's your opinion?

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-09-03 Thread Yugo Nagata
Hi, On Sat, 1 Sep 2018 07:40:40 +0200 (CEST) Fabien COELHO wrote: > > Attached is a patch to allow pg_verity_checksums to specify a database > > to scan. This is usefule for users who want to verify checksums of > > relations > > in a specific database. We can specify a database by OID using

Re: Stored procedures and out parameters

2018-09-03 Thread Chapman Flack
On 09/02/18 16:37, Robert Haas wrote: > On Thu, Aug 30, 2018 at 7:45 PM, Chapman Flack wrote: >> b to store in y. For any remote client, the result still needs to get >> back there before the client can apply any "this result gets assigned >> to my y variable" semantics, and is there any

Re: speeding up planning with partitions

2018-09-03 Thread Dilip Kumar
On Wed, Aug 29, 2018 at 5:36 PM, Amit Langote wrote: > It is more or less well known that the planner doesn't perform well with > more than a few hundred partitions even when only a handful of partitions > are ultimately included in the plan. Situation has improved a bit in PG > 11 where we

Re: [PATCH] Tab completion for ALTER DATABASE … SET TABLESPACE

2018-09-03 Thread Arthur Zakirov
Hello, On Thu, Aug 30, 2018 at 05:54:23PM +0100, Dagfinn Ilmari Mannsåker wrote: > ilm...@ilmari.org (Dagfinn Ilmari Mannsåker) writes: > > Hi hackers, > > > > I just noticed that psql's tab completion for ALTER TABLE … SET > > TABLESPACE was treating it as any other configuration parameter and >

Re: pg_verify_checksums failure with hash indexes

2018-09-03 Thread Dilip Kumar
On Mon, Sep 3, 2018 at 8:37 AM, Amit Kapila wrote: > On Sat, Sep 1, 2018 at 10:28 AM Dilip Kumar wrote: >> >> On Sat, Sep 1, 2018 at 8:22 AM, Robert Haas wrote: >> > On Thu, Aug 30, 2018 at 7:27 AM, Amit Kapila >> > wrote: >> > >> > I wouldn't bother bumping HASH_VERSION. First, the fix

Re: Pluggable Storage - Andres's take

2018-09-03 Thread Haribabu Kommi
On Tue, Aug 28, 2018 at 1:48 PM Haribabu Kommi wrote: > On Fri, Aug 24, 2018 at 12:50 PM Andres Freund wrote: > >> Hi, >> >> On 2018-08-24 11:55:41 +1000, Haribabu Kommi wrote: >> > On Tue, Aug 21, 2018 at 6:59 PM Andres Freund >> wrote: >> > >> > > On 2018-08-21 16:55:47 +1000, Haribabu Kommi

RE: libpq debug log

2018-09-03 Thread Iwata, Aya
> "Iwata, Aya" writes: > > I'm going to propose libpq debug log for analysis of queries on the > > application > side. > > I think that it is useful to determine whether the cause is on the > > application > side or the server side when a slow query occurs. > > Hm, how will you tell that