Re: Compiler warnings with --enable-dtrace

On Mon, May 7, 2018 at 9:42 AM, Tom Lane wrote: > Thomas Munro writes: >> --enable-dtrace produces compiler warnings about const correctness, >> except on macOS. That's because Apple's dtrace produces function >> declarations in probes.h that

Re: allow psql to watch \dt

Justin Pryzby writes: > I thought that would be desirable, although I don't see any better way of > getting there than this. Hm, but a lot of the \d commands involve more than one underlying query, as well as a bunch of postprocessing. I doubt that the approach you seem to

allow psql to watch \dt

I thought that would be desirable, although I don't see any better way of getting there than this. I don't see other commands for which which watch is wanted...but who am I to say that watching creation extention isn't useful? So I imagine this should be generalized to save query buffer for all

Re: PANIC during crash recovery of a recently promoted standby

On Fri, May 11, 2018 at 12:09:58PM -0300, Alvaro Herrera wrote: > Yeah, I had this exact comment, but I was unable to come up with a test > case that would cause a problem. pg_ctl promote would wait for the control file to be updated, so you cannot use it in the TAP tests to trigger the

Re: Having query cache in core

On 2018-05-12 08:20:13 +1000, CK Tan wrote: > On Sat, May 12, 2018 at 8:18 AM, Tatsuo Ishii wrote: > > > > > > > How do you handle tables hiding behind views? Also how does cached > > entries > > > in pgpools know if some tables are modified without going thru pgpool, eg > >

Re: Having query cache in core

On Sat, May 12, 2018 at 8:18 AM, Tatsuo Ishii wrote: > > > > How do you handle tables hiding behind views? Also how does cached > entries > > in pgpools know if some tables are modified without going thru pgpool, eg > > pgplsql or trigger or via psql directly? > > Pgpool-II

Re: Having query cache in core

>> > I think you need to know which tables are involved and if they were >> > modified. >> >> Of course. While creating a cache entry for a SELECT, we need to >> analyze it and extract tables involved in the SELECT. The information >> should be stored along with the cache entry. If any of the

Re: Having query cache in core

> On 11.05.2018 18:01, Tatsuo Ishii wrote: >> Plus checking username is neccessary (otherwise any user could >> retrieve a cache for a table lookup which is not permitted by other >> users). > > as the tables a cached query operated on is known anyway -- it's > needed > to purge cache entries

Re: perlcritic: Missing "return"

On 05/11/2018 09:59 AM, Mike Blackwell wrote: > After applying the perlcritic overrides Andrew used for the buildfarm, > one of the most common remaining level 4 warnings in the PostgreSQL > source, with 186 occurrences, is 'Subroutine does not end with "return"'. > > The point of this warning

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 10:20:17PM +0300, Teodor Sigaev wrote: > >>857f9c36cda520030381bd8c2af20adf0ce0e1d4 Skip full index scan during cleanup > >>of B-tree indexes when possible > > > >I read that and thought it was too details to be in the release notes. > >It is not that it is unimportant, but

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 12:28:57PM -0700, Peter Geoghegan wrote: > On Fri, May 11, 2018 at 12:17 PM, Peter Geoghegan wrote: > > * Suggest replacement sort item be phrased as: "Remove the > > configuration parameter replacement_sort_tuples. The > > replacement selection sort

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 12:17:47PM -0700, Peter Geoghegan wrote: > On Fri, May 11, 2018 at 12:04 PM, Andres Freund wrote: > > I don't think the table being 'append-only' is necessary? Nor does it > > have to be a manual vacuum. And 'needless index scan' sounds less than > >

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 12:17 PM, Peter Geoghegan wrote: > * Suggest replacement sort item be phrased as: "Remove the > configuration parameter replacement_sort_tuples. The > replacement selection sort algorithm is no longer used." Also, it should be moved to "Migration to Version

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 12:22:08PM -0700, Andres Freund wrote: > Btw, is it just me, or do the commit and docs confuse say stalled when > stale is intended? Should be fixed since yesterday's 8e12f4a250d250a89153da2eb9b91c31bb80c483 ? Justin

Re: Postgres 11 release notes

On 2018-05-11 15:11:31 -0400, Bruce Momjian wrote: > On Fri, May 11, 2018 at 04:00:58PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > OK, so what is the text that people will understand? This? > > > > > > Prevent manual VACUUMs on append-only tables from performing > > >

Re: Postgres 11 release notes

857f9c36cda520030381bd8c2af20adf0ce0e1d4 Skip full index scan during cleanup of B-tree indexes when possible I read that and thought it was too details to be in the release notes. It is not that it is unimportant, but it is hard to see how people would notice the difference or change their

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 12:04 PM, Andres Freund wrote: > I don't think the table being 'append-only' is necessary? Nor does it > have to be a manual vacuum. And 'needless index scan' sounds less than > it is/was, namely a full scan of the index. Perhaps something like: > >

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 11:59:12AM -0700, Andres Freund wrote: > Hi, > > Quick notes: > > > >Add Just-In-Time (JIT) compilation of plans > >run the by the executor > >(Andres Freund) > > > > > > It's currently not yet compilation of entire plans, but only

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 04:00:58PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > OK, so what is the text that people will understand? This? > > > > Prevent manual VACUUMs on append-only tables from performing > > needless index scans > > Make vacuum cheaper by avoiding

Re: Postgres 11 release notes

On 2018-05-11 14:59:04 -0400, Bruce Momjian wrote: > On Fri, May 11, 2018 at 11:50:51AM -0700, Andres Freund wrote: > > On 2018-05-11 14:44:06 -0400, Bruce Momjian wrote: > > > On Fri, May 11, 2018 at 07:49:50PM +0300, Teodor Sigaev wrote: > > > > > > > > > > > > Bruce Momjian wrote: > > > > >I

Re: Postgres 11 release notes

Andres Freund wrote: > > > >Add configure flag --with-llvm to test for > >LLVM support (Andres Freund) > > > > > >Have configure check for the availability of a C++ compiler > >(Andres Freund) > > > > I wonder if we shouldn't omit

Re: Postgres 11 release notes

Bruce Momjian wrote: > OK, so what is the text that people will understand? This? > > Prevent manual VACUUMs on append-only tables from performing > needless index scans Make vacuum cheaper by avoiding scans of btree indexes when not necessary ? Why "manual vacuum"? It's a

Re: Postgres 11 release notes

Hi, Quick notes: > >Add Just-In-Time (JIT) compilation of plans >run the by the executor >(Andres Freund) > > It's currently not yet compilation of entire plans, but only parts. I think that's a relevant distinction because I'd like to add that as a feature to

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 11:50:51AM -0700, Andres Freund wrote: > On 2018-05-11 14:44:06 -0400, Bruce Momjian wrote: > > On Fri, May 11, 2018 at 07:49:50PM +0300, Teodor Sigaev wrote: > > > > > > > > > Bruce Momjian wrote: > > > >I have committed the first draft of the Postgres 11 release notes.

Re: Postgres 11 release notes

On 2018-05-11 14:44:06 -0400, Bruce Momjian wrote: > On Fri, May 11, 2018 at 07:49:50PM +0300, Teodor Sigaev wrote: > > > > > > Bruce Momjian wrote: > > >I have committed the first draft of the Postgres 11 release notes. I > > >will add more markup soon. You can view the most current version

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 01:40:54PM -0400, Chapman Flack wrote: > On 05/11/2018 11:08 AM, Bruce Momjian wrote: > > > http://momjian.us/pgsql_docs/release-11.html > > > > I expect a torrent of feedback. ;-) > > Very superficial things: > > > Add predicate locking for Hash, GiST, and GIN

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 07:49:50PM +0300, Teodor Sigaev wrote: > > > Bruce Momjian wrote: > >I have committed the first draft of the Postgres 11 release notes. I > >will add more markup soon. You can view the most current version here: > > > > http://momjian.us/pgsql_docs/release-11.html >

Re: Clock with Adaptive Replacement

I have been working on caching as a hobby project for the last few years and would be happy to collaborate on an analysis and design. I've tackled a lot of these issues, wrote widely used implementations, and co-authored a paper for ACM's Transactions on Storage on an eviction policy. A short

Re: Postgres 11 release notes

On Fri, May 11, 2018 at 06:29:39PM +0200, Fabien COELHO wrote: > > Hello Bruce, > > > http://momjian.us/pgsql_docs/release-11.html > > >I expect a torrent of feedback. ;-) > > Here is some, for things I know about: > > >>Add major scripting features to pgbench (Fabien Coelho) > > I

Re: Postgres 11 release notes

On 05/11/2018 11:08 AM, Bruce Momjian wrote: > http://momjian.us/pgsql_docs/release-11.html > > I expect a torrent of feedback. ;-) Very superficial things: Add predicate locking for Hash, GiST, and GIN indexes s/likelyhood/likelihood/ Add extension jsonb_plpython There are two

Re: Postgres 11 release notes

Bruce Momjian wrote: I have committed the first draft of the Postgres 11 release notes. I will add more markup soon. You can view the most current version here: http://momjian.us/pgsql_docs/release-11.html I expect a torrent of feedback. ;-) Hi! Seems, you miss:

Re: Having query cache in core

On 11.05.2018 18:01, Tatsuo Ishii wrote: Plus checking username is neccessary (otherwise any user could retrieve a cache for a table lookup which is not permitted by other users). as the tables a cached query operated on is known anyway -- it's needed to purge cache entries when table content

Re: Postgres 11 release notes

Hello Bruce, http://momjian.us/pgsql_docs/release-11.html I expect a torrent of feedback. ;-) Here is some, for things I know about: Add major scripting features to pgbench (Fabien Coelho) I assume that you are refering to "bc7fa0c1". I do not think that anything qualifies

Re: Compiler warnings with --enable-dtrace

On Sat, May 5, 2018 at 6:22 AM, Thomas Munro wrote: > Hi hackers, > > --enable-dtrace produces compiler warnings about const correctness, > except on macOS. That's because Apple's dtrace produces function > declarations in probes.h that take strings as const char

Re: Having query cache in core

> that's almost actually how the MySQL query cache works: the query > cache > lookup kicks in even before the query is parsed, to get maximum gain > from cache hits. > > It does not just take the query text into account alone though, > but also the current default database, protocol version, >

Re: Having query cache in core

On Fri, May 11, 2018, 10:26 PM Tatsuo Ishii wrote: > > > > > I think you need to know which tables are involved and if they were > > modified. > > Of course. While creating a cache entry for a SELECT, we need to > analyze it and extract tables involved in the SELECT. The

Re: Having query cache in core

>> If any of the tables were modified, cache entries using the table must be > removed. >> (these are already implemented in Pgpool-II's in memory query cache) > > How do you identify updates made from a pl/pgsql procedure? Pgpool-II does not invalidate query cache in that case. I think in-core

Re: PANIC during crash recovery of a recently promoted standby

Michael Paquier wrote: > On Thu, May 10, 2018 at 10:52:12AM +0530, Pavan Deolasee wrote: > > I propose that we should always clear the minRecoveryPoint after promotion > > to ensure that crash recovery always run to the end if a just-promoted > > standby crashes before completing its first regular

Postgres 11 release notes

I have committed the first draft of the Postgres 11 release notes. I will add more markup soon. You can view the most current version here: http://momjian.us/pgsql_docs/release-11.html I expect a torrent of feedback. ;-) On a personal note, I want to apologize for not adequately

Re: [HACKERS] Surjective functional indexes

On 11 May 2018 at 16:37, Andres Freund wrote: > On 2018-05-11 14:56:12 +0200, Simon Riggs wrote: >> On 11 May 2018 at 05:32, Andres Freund wrote: >> > No. Simon just claimed it's not actually a concern: >> >

Re: pg_locale compilation error with Visual Studio 2017

Sandeep Thakkar writes: > I found the same error was raised by someone in pgsql-general lists but > don't see the submitted patch was committed. Here is the discussion link >

Re: [HACKERS] Surjective functional indexes

On 2018-05-11 14:56:12 +0200, Simon Riggs wrote: > On 11 May 2018 at 05:32, Andres Freund wrote: > > No. Simon just claimed it's not actually a concern: > > https://www.postgresql.org/message-id/canp8+j+vtskphep_gmqmeqdwakst2kbotee0yz-my+agh0a...@mail.gmail.com > > > > And

Re: [HACKERS] Surjective functional indexes

On Fri, May 11, 2018 at 4:58 AM, Konstantin Knizhnik < k.knizh...@postgrespro.ru> wrote: > > Sorry, may be I do not completely understand you. > So whats happed before this patch: > > - On update postgres compares old and new values of all changed attributes > to determine whether them are

perlcritic: Missing "return"

After applying the perlcritic overrides Andrew used for the buildfarm, one of the most common remaining level 4 warnings in the PostgreSQL source, with 186 occurrences, is 'Subroutine does not end with "return"'. The point of this warning is that, in Perl, falling off the end of a subroutine

Re: Considering signal handling in plpython again

On 11 May 2018 10:01:56 EEST, Hubert Zhang wrote: >2. Add a flag in hook function to indicate whether to call >Py_AddPendingCall. >This is straightforward.(I prefer it) Yeah, that's what I had in mind, too. A global bool variable that's set when you enter libpython, and

Re: Having query cache in core

On 11.05.2018 14:26, Tatsuo Ishii wrote: If any of the tables were modified, cache entries using the table must be removed. (these are already implemented in Pgpool-II's in memory query cache) ... and this is the expensive part in the MySQL implementation that has rendered the query cache

Re: [HACKERS] Surjective functional indexes

On 11 May 2018 at 05:32, Andres Freund wrote: > On 2018-05-10 23:25:58 -0400, Robert Haas wrote: >> On Thu, Mar 1, 2018 at 2:48 PM, Andres Freund wrote: >> > I still don't think, as commented upon by Tom and me upthread, that we >> > want this feature in

Re: Indexes on partitioned tables and foreign partitions

On 9 May 2018 at 17:33, Robert Haas wrote: > On Wed, May 9, 2018 at 11:20 AM, Simon Riggs wrote: >> On 9 May 2018 at 16:15, Robert Haas wrote: >>> On Wed, May 9, 2018 at 11:14 AM, Simon Riggs wrote:

Re: postgres_fdw: Oddity in pushing down inherited UPDATE/DELETE joins to remote servers

(2018/05/11 16:19), Amit Langote wrote: > On 2018/05/11 16:12, Amit Langote wrote: >> Just to clarify, does this problem only arise because there is a pushed >> down join involving the child? That is, does the problem only occur as of >> the following commit: >> >> commit

Re: postgres_fdw: Oddity in pushing down inherited UPDATE/DELETE joins to remote servers

Hi Amit, (2018/05/11 16:12), Amit Langote wrote: > On 2018/05/10 21:41, Etsuro Fujita wrote: >> I think the reason for that is: in that case we try to find the target >> foreign-join RelOptInfo using find_join_rel in postgresPlanDirectModify, >> but can't find it, because the given root is the

Re: Having query cache in core

> If any of the tables were modified, cache entries using the table must be removed. > (these are already implemented in Pgpool-II's in memory query cache) How do you identify updates made from a pl/pgsql procedure? Vladimir

Re: Having query cache in core

>> Thanks for the advice. But I rather thought about bypassing the raw >> parser and the planner. i.e. use the query string (or its hash) as the >> index of the query cache. >> > > I think you need to know which tables are involved and if they were > modified. Of course. While creating a cache

Re: [HACKERS] Surjective functional indexes

On 11.05.2018 07:48, David G. Johnston wrote: On Thursday, February 1, 2018, Konstantin Knizhnik > wrote: Old + New for check = 2 plus calculate again in index = 3 Yes, we have to calculate the value of index

Re: Having query cache in core

On 11.05.2018 11:12, Tatsuo Ishii wrote: Thanks for the advice. But I rather thought about bypassing the raw parser and the planner. i.e. use the query string (or its hash) as the index of the query cache. that's almost actually how the MySQL query cache works: the query cache lookup kicks in

Re: Needless additional partition check in INSERT?

On 11 May 2018 at 14:50, Amit Khandekar wrote: > On 10 May 2018 at 15:26, David Rowley wrote: >> Yeah, the comments do need work. In order to make it a bit easier to >> document I changed the way that check_partition_constr is set. This is >>

Re: Needless additional partition check in INSERT?

On 10 May 2018 at 15:26, David Rowley wrote: > Yeah, the comments do need work. In order to make it a bit easier to > document I changed the way that check_partition_constr is set. This is > now done with an if/else if/else clause for both COPY and INSERT. > >

Re: Having query cache in core

On Fri, May 11, 2018, 7:13 PM Tatsuo Ishii wrote: > > You could probably write an extension for that, though. I think the > > planner hook and custom scans give you enough flexibility to do that > > without modifying the server code. > > Thanks for the advice. But I rather

Re: Having query cache in core

> You could probably write an extension for that, though. I think the > planner hook and custom scans give you enough flexibility to do that > without modifying the server code. Thanks for the advice. But I rather thought about bypassing the raw parser and the planner. i.e. use the query string

Re: [HACKERS] asynchronous execution

Hello. This is the new version of $Subject. But, this is not just a rebased version. On the way fixing serious conflicts, I refactored patch and I believe this becomes way readable than the previous shape. # 0003 lacks changes of postgres_fdw.out now. - Waiting queue manipulation is moved into

Re: postgres_fdw: Oddity in pushing down inherited UPDATE/DELETE joins to remote servers

On 2018/05/11 16:12, Amit Langote wrote: > Just to clarify, does this problem only arise because there is a pushed > down join involving the child? That is, does the problem only occur as of > the following commit: > > commit 1bc0100d270e5bcc980a0629b8726a32a497e788 > Author: Robert Haas

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

On Thu, May 10, 2018 at 6:23 PM, Etsuro Fujita wrote: > (2018/05/10 13:04), Ashutosh Bapat wrote: >> >> On Wed, May 9, 2018 at 5:20 PM, Etsuro Fujita >> wrote: >>> >>> (2018/04/25 18:51), Ashutosh Bapat wrote: Actually I

Re: postgres_fdw: Oddity in pushing down inherited UPDATE/DELETE joins to remote servers

Fujita-san, On 2018/05/10 21:41, Etsuro Fujita wrote: > I think the reason for that is: in that case we try to find the target > foreign-join RelOptInfo using find_join_rel in postgresPlanDirectModify, > but can't find it, because the given root is the *parent* root and > doesn't have join

Re: Considering signal handling in plpython again

Thanks Heikki and Robert for your comments. I reviewed Heikki's patch and let's enhance it. As Heikki mentioned, there is a problem when no Python code is being executed. I tested it in the following case "select pysleep();" and then type ctrl-c, query cancelled successfully.(Patch works:))

Re: Needless additional partition check in INSERT?

On 2018/05/11 15:27, Michael Paquier wrote: > That's really up to the patch > author at the end (I prefer matching with NULL, but usually it is better > to comply with the surroundings for consistency). Yeah. I think in this case I'll have to withdraw my comment because most places that check

Re: Should we add GUCs to allow partition pruning to be disabled?

On Fri, May 11, 2018 at 12:59:27PM +0900, Amit Langote wrote: > On 2018/05/11 2:13, Robert Haas wrote: >> On Thu, May 10, 2018 at 12:58 PM, Alvaro Herrera >> wrote: >>> David G. Johnston wrote: As a user I don't really need to know which model is implemented and the

Re: Needless additional partition check in INSERT?

On 2018/05/11 15:12, David Rowley wrote: > Thanks for looking > > On 11 May 2018 at 17:48, Amit Langote wrote: >> By the way, >> >> +!resultRelInfo->ri_PartitionRoot) >> >> This should be resultRelInfo->ri_PartitionRoot == NULL, because the above >>

Re: Needless additional partition check in INSERT?

On Fri, May 11, 2018 at 06:12:38PM +1200, David Rowley wrote: > On 11 May 2018 at 17:48, Amit Langote wrote: >> By the way, >> >> +!resultRelInfo->ri_PartitionRoot) >> >> This should be resultRelInfo->ri_PartitionRoot == NULL, because the above >> gives

Re: Needless additional partition check in INSERT?

Thanks for looking On 11 May 2018 at 17:48, Amit Langote wrote: > By the way, > > +!resultRelInfo->ri_PartitionRoot) > > This should be resultRelInfo->ri_PartitionRoot == NULL, because the above > gives an impression that ri_PartitionRoot is a Boolean.