Re: PG 12 draft release notes
On 5/12/19 11:42 PM, Bruce Momjian wrote: > On Sun, May 12, 2019 at 10:49:07AM -0400, Jonathan Katz wrote: >> Hi Bruce, >> >> On 5/11/19 4:33 PM, Bruce Momjian wrote: >>> I have posted a draft copy of the PG 12 release notes here: >>> >>> http://momjian.us/pgsql_docs/release-12.html >>> >>> They are committed to git. It includes links to the main docs, where >>> appropriate. Our official developer docs will rebuild in a few hours. >> >> Thank you for working on this, I know it's a gargantuan task. >> >> I have a small modification for a section entitled "Source Code" which >> is a repeat of the previous section. Based on the bullet points in that >> part, I thought "Documentation" might be a more appropriate name; please >> see attached. > > Yes, I saw that myself and just updated it. Thanks. Great! There is a placeholder for PG_COLORS: This is enabled with by setting environment variable PG_COLORS. EXAMPLE? I've attached the following that provides an example of how to use this environmental variable. Thanks! Jonathan diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml index 8b668ade0f..e05aacaf3c 100644 --- a/doc/src/sgml/release-12.sgml +++ b/doc/src/sgml/release-12.sgml @@ -2479,7 +2479,9 @@ Author: Peter Eisentraut This is enabled with by setting environment variable - PG_COLORS. EXAMPLE? + PG_COLORS using ANSI escape codes for colors. For example, + the default setting is + PG_COLORS="error=01;31:warning=01;35:locus=01" signature.asc Description: OpenPGP digital signature
Re: PG 12 draft release notes
On 9/2/19 1:37 PM, Andrey Borodin wrote: > > >> 2 сент. 2019 г., в 22:02, Jonathan S. Katz написал(а): >> >> >> Attached is a patch proposing items for the major items section. This is >> working off of the ongoing draft of the press release[1]. Feedback >> welcome. With respect to the linking, I tried I to give a bunch of >> jumping off points for users to explore the features, but visually tried >> to ensure the readability was consistent. > > + > + Reduction of WAL overhead of > + GiST, GIN, > and > + SP-GiST indexes and added support > + for covering indexes via the > + INCLUDE clause for > + SP-GiST indexes > + > > Maybe I'm missing something, but covering indexes are supported in GiST, not > in SP-GiST. Nope, you're absolutely correct: that was a typo as a result of copying/pasting formatting. Attached is a revision that correctly specifies covering indexes for GiST. Thanks! Jonathan diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml index 8b668ade0f..62ab9fb21f 100644 --- a/doc/src/sgml/release-12.sgml +++ b/doc/src/sgml/release-12.sgml @@ -21,7 +21,125 @@ - big item + + General performance improvements, including: + + + + Optimizations to space utilization and read/write performance for + B-tree indexes that are frequently modified + + + + + Partitioning enhancements that include query performance improvements + on tables with thousands of partitions, improved performance with + and , and the + ability to execute + ATTACH PARTITION without blocking queries + + + + + Common table expressions + (CTE) can now be automatically inlined + + + + + Reduction of WAL overhead of + GiST, GIN, and + SP-GiST indexes and added support + for covering indexes via the + INCLUDE clause for + GiST indexes + + + + + now supports most-common value + (MCV) statistics to support improved query plans for columns that are + nonuniformly distributed + + + + + + + + + Enhancements to administrative functionality, including: + + + + Rebuild an index without blocking writes to a table with the + + REINDEX CONCURRENTLY command + + + + + can enable/disable page checksums, + used for detecting data corruption, in an offline cluster + + + Progress reporting statistics for , + , , + VACUUM FULL, and + + + + + + + + + + Support for the SQL/JSON + path language + + + + + + Stored generated columns + + + + + + Nondeterministic ICU + collations that enable + case-insensitive and accent-insensitive ordering + + + + + + New authentication features, including: + + + + Client and server-side encryption for authentication over the + GSSAPI + + + + + Discovery of LDAP servers if PostgreSQL is + built with OpenLDAP + + + + + Support multifactor authentication with the + clientcert=verify-full option when combined + with an additional authentication method in + pg_hba.conf + + + + signature.asc Description: OpenPGP digital signature
Re: PG 12 draft release notes
> 2 сент. 2019 г., в 22:02, Jonathan S. Katz написал(а): > > > Attached is a patch proposing items for the major items section. This is > working off of the ongoing draft of the press release[1]. Feedback > welcome. With respect to the linking, I tried I to give a bunch of > jumping off points for users to explore the features, but visually tried > to ensure the readability was consistent. + + Reduction of WAL overhead of + GiST, GIN, and + SP-GiST indexes and added support + for covering indexes via the + INCLUDE clause for + SP-GiST indexes + Maybe I'm missing something, but covering indexes are supported in GiST, not in SP-GiST. Best regards, Andrey Borodin.
Re: PG 12 draft release notes
On 5/11/19 4:33 PM, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. > Thanks again for compiling and writing up the release notes. I know it is no small effort. Attached is a patch proposing items for the major items section. This is working off of the ongoing draft of the press release[1]. Feedback welcome. With respect to the linking, I tried I to give a bunch of jumping off points for users to explore the features, but visually tried to ensure the readability was consistent. I also attached a patch addressing the "MENTION ITS AFFECT ON ORDERING?" in which I choose to answer it "yes" and added a comment :) Thanks, Jonathan [1] https://www.postgresql.org/message-id/c56eeb88-4a8c-2c6c-b5f1-9d46872c247c%40postgresql.org diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml index 8b668ade0f..e5929f887c 100644 --- a/doc/src/sgml/release-12.sgml +++ b/doc/src/sgml/release-12.sgml @@ -21,7 +21,125 @@ - big item + + General performance improvements, including: + + + + Optimizations to space utilization and read/write performance for + B-tree indexes that are frequently modified + + + + + Partitioning enhancements that include query performance improvements + on tables with thousands of partitions, improved performance with + and , and the + ability to execute + ATTACH PARTITION without blocking queries + + + + + Common table expressions + (CTE) can now be automatically inlined + + + + + Reduction of WAL overhead of + GiST, GIN, and + SP-GiST indexes and added support + for covering indexes via the + INCLUDE clause for + SP-GiST indexes + + + + + now supports most-common value + (MCV) statistics to support improved query plans for columns that are + nonuniformly distributed + + + + + + + + + Enhancements to administrative functionality, including: + + + + Rebuild an index without blocking writes to a table with the + + REINDEX CONCURRENTLY command + + + + + can enable/disable page checksums, + used for detecting data corruption, in an offline cluster + + + Progress reporting statistics for , + , , + VACUUM FULL, and + + + + + + + + + + Support for the SQL/JSON + path language + + + + + + Stored generated columns + + + + + + Nondeterministic ICU + collations that enable + case-insensitive and accent-insensitive ordering + + + + + + New authentication features, including: + + + + Client and server-side encryption for authentication over the + GSSAPI + + + + + Discovery of LDAP servers if PostgreSQL is + built with OpenLDAP + + + + + Support multifactor authentication with the + clientcert=verify-full option when combined + with an additional authentication method in + pg_hba.conf + + + + diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml index 8b668ade0f..20ca5eb569 100644 --- a/doc/src/sgml/release-12.sgml +++ b/doc/src/sgml/release-12.sgml @@ -1941,9 +1941,9 @@ Author: Peter Eisentraut - This allows case and accent-agnostic equality comparisons using - "nondeterministic" collations. This is only supported for - ICU collations. MENTION ITS AFFECT ON ORDERING? + This allows case and accent-agnostic equality comparisons, such as during + an ordering operation, using "nondeterministic" collations. This is only + supported for ICU collations. signature.asc Description: OpenPGP digital signature
Re: PG 12 draft release notes
On Thu, Jul 25, 2019 at 6:37 PM Bruce Momjian wrote: > Attached patch applied, thanks. Thanks Bruce, -- Peter Geoghegan
Re: PG 12 draft release notes
On Mon, Jul 15, 2019 at 06:21:59PM -0700, Peter Geoghegan wrote: > On Wed, Jun 12, 2019 at 7:48 PM Bruce Momjian wrote: > > Great, patch applied. > > I think that it would make sense to have a v12 release note item for > amcheck's new "rootdescend" verification option: > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c1afd175b5b2e5c44f6da34988342e00ecdfb518 > > It is a user facing feature, which increments the amcheck extension > version number. Attached patch applied, thanks. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml new file mode 100644 index 4c55656..6b54a9e *** a/doc/src/sgml/release-12.sgml --- b/doc/src/sgml/release-12.sgml *** Author: Tom Lane *** 3228,3233 --- 3228,3245 + + +Add a parameter to a function to check +each index tuple from the root of the tree. + + + + +
Re: PG 12 draft release notes
On Tue, Jul 16, 2019 at 03:26:31PM +0900, Michael Paquier wrote: > On Mon, Jul 15, 2019 at 08:51:34PM +0200, Laurenz Albe wrote: > > I wonder if commits 0ba06e0bf and 40cfe8606 are worth mentioning > > in the release notes. They make "pg_test_fsync" work correctly > > on Windows for the first time. > > I don't know about this point specifically. Improving support for > pg_test_fsync on Windows is just a side effect of the first commit > which benefits all frontend tools (the second commit is an > embarrassing bug fix for the first one). And at the same time we > don't really add in the release notes low-level improvements like > these ones. Well, if we were reporting incorrect results before, that seems like a fix, with updated wording, of course, to mention just the fix. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Mon, Jul 15, 2019 at 08:51:34PM +0200, Laurenz Albe wrote: > On Sat, 2019-05-11 at 16:33 -0400, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > I wonder if commits 0ba06e0bf and 40cfe8606 are worth mentioning > in the release notes. They make "pg_test_fsync" work correctly > on Windows for the first time. Oh, I missed adding that. I applied the attached patch, with updated wording, and moved it to the Server Applications section. Thanks. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml new file mode 100644 index 57365c5..4c55656 *** a/doc/src/sgml/release-12.sgml --- b/doc/src/sgml/release-12.sgml *** Author: Andrew Dunstan + + + + +Fix to report accurate +open_datasync durations on +Windows (Laurenz Albe) + + +
Re: PG 12 draft release notes
On Mon, Jul 15, 2019 at 08:51:34PM +0200, Laurenz Albe wrote: > I wonder if commits 0ba06e0bf and 40cfe8606 are worth mentioning > in the release notes. They make "pg_test_fsync" work correctly > on Windows for the first time. I don't know about this point specifically. Improving support for pg_test_fsync on Windows is just a side effect of the first commit which benefits all frontend tools (the second commit is an embarrassing bug fix for the first one). And at the same time we don't really add in the release notes low-level improvements like these ones. -- Michael signature.asc Description: PGP signature
Re: PG 12 draft release notes
On Wed, Jun 12, 2019 at 7:48 PM Bruce Momjian wrote: > Great, patch applied. I think that it would make sense to have a v12 release note item for amcheck's new "rootdescend" verification option: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c1afd175b5b2e5c44f6da34988342e00ecdfb518 It is a user facing feature, which increments the amcheck extension version number. Thanks -- Peter Geoghegan
Re: PG 12 draft release notes
On Mon, Jul 15, 2019 at 10:18:07PM +0700, John Naylor wrote: > I noticed a small typo in the release notes in the list of languages > with new stemmers (see attached) Sorry, fixed, thanks. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Sat, 2019-05-11 at 16:33 -0400, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: I wonder if commits 0ba06e0bf and 40cfe8606 are worth mentioning in the release notes. They make "pg_test_fsync" work correctly on Windows for the first time. Yours, Laurenz Albe diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml index 71f60d6961..e69d4955ce 100644 --- a/doc/src/sgml/release-12.sgml +++ b/doc/src/sgml/release-12.sgml @@ -2965,10 +2965,24 @@ Author: Andres Freund Require a C99-supported compiler, and MSVC 2013 or later on Windows (Andres Freund) + + + + + Use the same functions to open files in frontend and backend code + on Windows, thus fixing the "open_datasync" + test in "pg_test_fsync" (Laurenz Albe) + + +
Re: PG 12 draft release notes
I noticed a small typo in the release notes in the list of languages with new stemmers (see attached) -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services stemming-Tamil.patch Description: Binary data
Re: PG 12 draft release notes
On Thu, Jun 13, 2019 at 09:12:58AM -0400, Bruce Momjian wrote: > I went with this wording: > > This prevents unauthorized locking, which could interfere with > user queries. Okay, fine for me. Thanks for updating the notes. -- Michael signature.asc Description: PGP signature
Re: PG 12 draft release notes
On Thu, Jun 13, 2019 at 09:11:08AM -0400, Bruce Momjian wrote: > On Thu, Jun 13, 2019 at 03:33:48PM +0900, Michael Paquier wrote: > > On Wed, Jun 12, 2019 at 05:25:37PM -0400, Bruce Momjian wrote: > > > Since we did not backpatch this fix, I am hesitant to spell out exactly > > > how to exploit this DOS attack. Yes, people can read it in the email > > > archives, and commit messages, but I don't see the value in spelling it > > > out the release notes too. > > > > We could go for a more general version of that, for the reason that it > > involves all relations, like: > > "A caller of TRUNCATE or VACUUM could previously queue for an access > > exclusive lock on a relation it may not have permission to truncate or > > vacuum, leading to relations to be blocked from being accessed." > > Uh, that still seems to suggest an attack and I am not sure that > information is very useful to users. I went with this wording: This prevents unauthorized locking, which could interfere with user queries. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Thu, Jun 13, 2019 at 03:33:48PM +0900, Michael Paquier wrote: > On Wed, Jun 12, 2019 at 05:25:37PM -0400, Bruce Momjian wrote: > > Since we did not backpatch this fix, I am hesitant to spell out exactly > > how to exploit this DOS attack. Yes, people can read it in the email > > archives, and commit messages, but I don't see the value in spelling it > > out the release notes too. > > We could go for a more general version of that, for the reason that it > involves all relations, like: > "A caller of TRUNCATE or VACUUM could previously queue for an access > exclusive lock on a relation it may not have permission to truncate or > vacuum, leading to relations to be blocked from being accessed." Uh, that still seems to suggest an attack and I am not sure that information is very useful to users. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Wed, Jun 12, 2019 at 05:25:37PM -0400, Bruce Momjian wrote: > Since we did not backpatch this fix, I am hesitant to spell out exactly > how to exploit this DOS attack. Yes, people can read it in the email > archives, and commit messages, but I don't see the value in spelling it > out the release notes too. We could go for a more general version of that, for the reason that it involves all relations, like: "A caller of TRUNCATE or VACUUM could previously queue for an access exclusive lock on a relation it may not have permission to truncate or vacuum, leading to relations to be blocked from being accessed." -- Michael signature.asc Description: PGP signature
Re: PG 12 draft release notes
On Wed, Jun 12, 2019 at 07:42:31PM -0700, Peter Geoghegan wrote: > On Wed, Jun 12, 2019 at 7:29 PM Bruce Momjian wrote: > > OK, I mentioned something about increased locality now. Patch attached. > > Looks good -- locality is a good catch-all term. Great, patch applied. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Wed, Jun 12, 2019 at 7:29 PM Bruce Momjian wrote: > OK, I mentioned something about increased locality now. Patch attached. Looks good -- locality is a good catch-all term. Thanks! -- Peter Geoghegan
Re: PG 12 draft release notes
On Wed, Jun 12, 2019 at 06:34:27PM -0700, Peter Geoghegan wrote: > > I was wrong. I was thinking of this commit: > > > > commit d2086b08b0 > > Author: Alexander Korotkov > > Date: Sat Jul 28 00:31:40 2018 +0300 > > > > Reduce path length for locking leaf B-tree pages during insertion > > > > If you had to cut one thing from this list, then I would suggest that > > > it be this item. It's nice, but it's also very obvious, which makes it > > > hard to explain. > > > Right. The commit mentioned a 4.5x speedup in a rare benchmark, so I > > added it lower on the list. > > My remark about cutting an item referred to a lesser item that I > worked on (the 'Add nbtree high key "continuescan" optimization' > commit), not Alexander independent B-Tree work. I think that > Alexander's optimization is also quite effective. Though FWIW the 4.5x > improvement concerned a case involving lots of duplicates...cases with > a lot of duplicates will be far far better in Postgres 12. (I never > tested my patch without Alexander's commit, since it went in early in > the v12 cycle.) OK, good to know. > > Attached is an updated patch. I might have missed something, but I > > think it might be close. > > This looks great. I do have a few things: > > * I would put "Improve performance and space utilization of btree > indexes with many duplicates" first (before "Allow multi-column btree > indexes to be smaller"). I think that this is far more common than we > tend to assume, and is also where the biggest benefits are. OK, done, I was wondering about that. > * The wording of the "many duplicates" item itself is almost perfect, > though the "...and inefficiency when VACUUM needed to find a row for > removal" part seems a bit off -- this is really about the > effectiveness of VACUUM, not the speed at which the VACUUM completes > (it's a bit faster, but that's not that important). Perhaps that part > should read: "...and often failed to efficiently recycle space made > available by VACUUM". Something like that. Ah, I see what you mean --- recycle entire pages. I have updated the patch. > * The "Allow multi-column btree indexes to be smaller" item is about > both suffix truncation, and about the "Split after new tuple" > optimization. I think that that makes it more complicated than it > needs to be. While the improvements that we saw with TPC-C on account > of the "Split after new tuple" optimization were nice, I doubt that > users will be looking out for it. I would be okay if you dropped any > mention of the "Split after new tuple" optimization, in the interest > of making the description more useful to users. We can just lose that. OK, done. > * Once you simplify the item by making it all about suffix truncation, > it would make sense to change the single line summary to "Reduce the > number of branch blocks needed for multi-column indexes". Then go on > to talk about how we now only store those columns that are necessary > to guide index scans in tuples stored in branch pages (we tend to call > branch pages internal pages, but branch pages seems friendlier to me). > Note that the user docs of other database systems reference these > details, even in their introductory material on how B-Tree indexes > work. The term "suffix truncation" isn't something users have heard > of, and we shouldn't use it here, but the *idea* of suffix truncation > is very well established. As I mentioned, it matters for things like > covering indexes (indexes that are designed to be used by index-only > scans, which are not necessarily INCLUDE indexes). OK, I mentioned something about increased locality now. Patch attached. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml new file mode 100644 index 4a6c989..fcc49ff *** a/doc/src/sgml/release-12.sgml --- b/doc/src/sgml/release-12.sgml *** Author: Tom Lane *** 606,627 ! Improve speed of btree index insertions (Peter Geoghegan, ! Alexander Korotkov) ! The new code improves the space-efficiency of page splits, ! reduces locking overhead, and gives better performance for ! UPDATEs and DELETEs on ! indexes with many duplicates. --- 606,669 ! Improve performance and space utilization of btree indexes with ! many duplicates (Peter Geoghegan, Heikki Linnakangas) ! Previously, duplicate index entries were stored unordered within ! their duplicate groups. This caused overhead during index ! inserts, wasted space due to excessive page splits, and reduced ! VACUUM's ability to recycle entire
Re: PG 12 draft release notes
On Wed, Jun 12, 2019 at 5:22 PM Bruce Momjian wrote: > I had become so confused by this item that I needed a few weeks to > settle on what was actually going on. I put a lot of time into my pgCon talk, especially on the diagrams. Seems like that paid off. Even Heikki was confused by my explanations at one point. I should go add a similar diagram to our documentation, under "Chapter 63. B-Tree Indexes", because diagrams are the only sensible way to explain the concepts. > I was wrong. I was thinking of this commit: > > commit d2086b08b0 > Author: Alexander Korotkov > Date: Sat Jul 28 00:31:40 2018 +0300 > > Reduce path length for locking leaf B-tree pages during insertion > > If you had to cut one thing from this list, then I would suggest that > > it be this item. It's nice, but it's also very obvious, which makes it > > hard to explain. > Right. The commit mentioned a 4.5x speedup in a rare benchmark, so I > added it lower on the list. My remark about cutting an item referred to a lesser item that I worked on (the 'Add nbtree high key "continuescan" optimization' commit), not Alexander independent B-Tree work. I think that Alexander's optimization is also quite effective. Though FWIW the 4.5x improvement concerned a case involving lots of duplicates...cases with a lot of duplicates will be far far better in Postgres 12. (I never tested my patch without Alexander's commit, since it went in early in the v12 cycle.) > Yes, locality. "Locality" is one of my favorite words. > Attached is an updated patch. I might have missed something, but I > think it might be close. This looks great. I do have a few things: * I would put "Improve performance and space utilization of btree indexes with many duplicates" first (before "Allow multi-column btree indexes to be smaller"). I think that this is far more common than we tend to assume, and is also where the biggest benefits are. * The wording of the "many duplicates" item itself is almost perfect, though the "...and inefficiency when VACUUM needed to find a row for removal" part seems a bit off -- this is really about the effectiveness of VACUUM, not the speed at which the VACUUM completes (it's a bit faster, but that's not that important). Perhaps that part should read: "...and often failed to efficiently recycle space made available by VACUUM". Something like that. * The "Allow multi-column btree indexes to be smaller" item is about both suffix truncation, and about the "Split after new tuple" optimization. I think that that makes it more complicated than it needs to be. While the improvements that we saw with TPC-C on account of the "Split after new tuple" optimization were nice, I doubt that users will be looking out for it. I would be okay if you dropped any mention of the "Split after new tuple" optimization, in the interest of making the description more useful to users. We can just lose that. * Once you simplify the item by making it all about suffix truncation, it would make sense to change the single line summary to "Reduce the number of branch blocks needed for multi-column indexes". Then go on to talk about how we now only store those columns that are necessary to guide index scans in tuples stored in branch pages (we tend to call branch pages internal pages, but branch pages seems friendlier to me). Note that the user docs of other database systems reference these details, even in their introductory material on how B-Tree indexes work. The term "suffix truncation" isn't something users have heard of, and we shouldn't use it here, but the *idea* of suffix truncation is very well established. As I mentioned, it matters for things like covering indexes (indexes that are designed to be used by index-only scans, which are not necessarily INCLUDE indexes). Thanks! -- Peter Geoghegan
Re: PG 12 draft release notes
On Wed, Jun 12, 2019 at 03:06:34PM -0700, Peter Geoghegan wrote: > On Wed, Jun 12, 2019 at 1:16 PM Bruce Momjian wrote: > > First, my apologies in getting to this so late. Peter Geoghegan > > supplied me with slides and a video to study, and I now understand how > > complex the btree improvements are. Here is a video of Peter's > > presentation at PGCon: > > > > https://www.youtube.com/watch?v=p5RaATILoiE > > Thank you for going to the trouble of researching the B-Tree stuff in > detail! I wouldn't ask that of anybody in the position of writing > release notes, so it's appreciated. It is awkward to take the work > that I've done and make it into multiple bullet points; I have a hard > time with that myself. I had become so confused by this item that I needed a few weeks to settle on what was actually going on. > > The over-arching improvement to btree in PG 12 is the ordering of index > > entries by tid so all entries are unique. > > Right. Everything good happens as a direct or indirect result of the > TID-as-column thing. That is really the kernel of the whole thing, > because it means that the implementation now *fully* follows the > Lehman and Yao design. Right. > > 1. Since all index tuples are ordered, you can move from one leaf page > > to the next without keeping a lock on the internal page that references > > it, increasing concurrency. > > I'm not sure what you mean here. We've never had to keep a lock on an > internal page while holding a lock on a leaf page -- such "lock > coupling" was necessary in earlier B-Tree designs, but Lehman and > Yao's algorithm avoids that. Of course, that isn't new. > > I think that you're talking about the way that we now check the high > key during index scans, and find that we don't have to move to the > next leaf page, per this commit: > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=29b64d1de7c77ffb5cb10696693e6ed8a6fc481c I was wrong. I was thinking of this commit: commit d2086b08b0 Author: Alexander Korotkov Date: Sat Jul 28 00:31:40 2018 +0300 Reduce path length for locking leaf B-tree pages during insertion In our B-tree implementation appropriate leaf page for new tuple insertion is acquired using _bt_search() function. This function always returns leaf page locked in shared mode. In order to obtain exclusive lock, caller have to relock the page. This commit makes _bt_search() function lock leaf page immediately in exclusive mode when needed. That removes unnecessary relock and, in turn reduces lock contention for B-tree leaf pages. Our experiments on multi-core systems showed acceleration up to 4.5 times in corner case. but got it confused by an optimization you mentioned in the video where you were talking about the need to perhaps recheck the internal page when moving right. We certainly don't keep the internal page locked. > All of the suffix truncation stuff is good because it makes separator > keys in internal pages smaller, but it's also good because the > separator keys are simultaneously more "discriminating". We tend to > get a nice "clean break" between leaf pages, so checking the high key > before moving right to find additional matches (once we've already > returned some tuples to the executor) is surprisingly effective. It > would have been possible to add this optimization even without the > suffix truncation stuff, but truncation makes it effective. > > If you had to cut one thing from this list, then I would suggest that > it be this item. It's nice, but it's also very obvious, which makes it > hard to explain. I mean, why should there be any ambiguity at all? > Unless we have to return *hundreds* of items to the index scan, then a > simple "select * from foo where bar = ?" style query should only have > to visit one leaf page, even when the constant happens to be on the > boundary of a leaf page (maybe a concurrent page split could make this > happen, but that should be rare). Right. The commit mentioned a 4.5x speedup in a rare benchmark, so I added it lower on the list. > > 2. When inserting a duplicate value in the index, we used to try a few > > pages to see if there was space, then "get tired" and just split a page > > containing duplicates. Now that there are no duplicates (because > > duplicate key fields are sorted by tid) the system knows exactly what > > page the index tuple belongs on, and inserts or splits the page as > > necessary. > > Right -- inserts will descend straight to the correct leaf page, and > the "getting tired" dance isn't used anymore. This makes insertions > faster, but more importantly is a better high level strategy for > storing lots of duplicates. We'll dirty far fewer pages, because > insertions automatically end up inserting around the same place as we > inserted to a moment ago. Insertions of duplicates behave like > serial/auto-incrementing insertions, which was already >
Re: PG 12 draft release notes
On Wed, Jun 12, 2019 at 1:16 PM Bruce Momjian wrote: > First, my apologies in getting to this so late. Peter Geoghegan > supplied me with slides and a video to study, and I now understand how > complex the btree improvements are. Here is a video of Peter's > presentation at PGCon: > > https://www.youtube.com/watch?v=p5RaATILoiE Thank you for going to the trouble of researching the B-Tree stuff in detail! I wouldn't ask that of anybody in the position of writing release notes, so it's appreciated. It is awkward to take the work that I've done and make it into multiple bullet points; I have a hard time with that myself. > The over-arching improvement to btree in PG 12 is the ordering of index > entries by tid so all entries are unique. Right. Everything good happens as a direct or indirect result of the TID-as-column thing. That is really the kernel of the whole thing, because it means that the implementation now *fully* follows the Lehman and Yao design. > 1. Since all index tuples are ordered, you can move from one leaf page > to the next without keeping a lock on the internal page that references > it, increasing concurrency. I'm not sure what you mean here. We've never had to keep a lock on an internal page while holding a lock on a leaf page -- such "lock coupling" was necessary in earlier B-Tree designs, but Lehman and Yao's algorithm avoids that. Of course, that isn't new. I think that you're talking about the way that we now check the high key during index scans, and find that we don't have to move to the next leaf page, per this commit: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=29b64d1de7c77ffb5cb10696693e6ed8a6fc481c All of the suffix truncation stuff is good because it makes separator keys in internal pages smaller, but it's also good because the separator keys are simultaneously more "discriminating". We tend to get a nice "clean break" between leaf pages, so checking the high key before moving right to find additional matches (once we've already returned some tuples to the executor) is surprisingly effective. It would have been possible to add this optimization even without the suffix truncation stuff, but truncation makes it effective. If you had to cut one thing from this list, then I would suggest that it be this item. It's nice, but it's also very obvious, which makes it hard to explain. I mean, why should there be any ambiguity at all? Unless we have to return *hundreds* of items to the index scan, then a simple "select * from foo where bar = ?" style query should only have to visit one leaf page, even when the constant happens to be on the boundary of a leaf page (maybe a concurrent page split could make this happen, but that should be rare). > 2. When inserting a duplicate value in the index, we used to try a few > pages to see if there was space, then "get tired" and just split a page > containing duplicates. Now that there are no duplicates (because > duplicate key fields are sorted by tid) the system knows exactly what > page the index tuple belongs on, and inserts or splits the page as > necessary. Right -- inserts will descend straight to the correct leaf page, and the "getting tired" dance isn't used anymore. This makes insertions faster, but more importantly is a better high level strategy for storing lots of duplicates. We'll dirty far fewer pages, because insertions automatically end up inserting around the same place as we inserted to a moment ago. Insertions of duplicates behave like serial/auto-incrementing insertions, which was already fast/well-optimized in various ways. It's easy to measure this by looking at index bloat when inserting lots of duplicates -- I came up with the 16% figure in the talk based on a simple insert-only test. > 3. Pivot tuples are used on internal pages and as min/max indicators on > leaf pages. These pivot tuples are now trimmed if their trailing key > fields are not significant. For example, if an index is > field1/field2/field3, and the page contains values for which field1==5 > and none that field1==6, there is no need to include field2 and field3 > in the pivot tuple --- it can just list the pivot as field1==5, > field2=infinity. This is called suffix truncation. Right -- that's exactly how it works. Users may find that indexes with lots of extra columns at the end won't get so bloated in Postgres 12. Indexing many columns is typically seen when index-only scans are important. Of course, you could have made those indexes INCLUDE indexes on v11, which is actually a closely related idea, but that makes it impossible to use the trailing/suffix columns in an ORDER BY. And, you have to know about INCLUDE indexes, and remember to use them. (This must be why Oracle can get away with not having INCLUDE indexes.) > Page splits used to just split the page in half, which minimizes the > number of page splits, but sometimes causes lots of wasted space. The > new code tries to split to reduce the
Re: PG 12 draft release notes
On Fri, Jun 7, 2019 at 12:04:33PM +0200, Adrien Nayrat wrote: > On 5/11/19 10:33 PM, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main docs, where > > appropriate. Our official developer docs will rebuild in a few hours. > > > > Hello, > > By looking to a user request to add greek in our FTS [1], I suggest to mention > which languages has been added in fd582317e. > > Patch attached. > > I hesitate to also mention these changes? > > > These all work in UTF8, and the indonesian and irish ones also work in > > LATIN1. > > > The non-UTF8 version of the hungarian stemmer now works in LATIN2 not > > LATIN1. > > > 1: > https://www.postgresql.org/message-id/trinity-f09793a1-8c13-4b56-94fe-10779e96c87e-1559896268438%403c-app-mailcom-bs16 Good idea, done. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Tue, May 28, 2019 at 08:58:23AM -0700, Andres Freund wrote: > > > > Add function > linkend="functions-info-partition">pg_partition_root() > to return top-most parent of a partition tree (Michaël Paquier) > > > > > > > > Add function > linkend="functions-info-partition">pg_partition_ancestors() > to report all ancestors of a partition (Álvaro Herrera) > > > > > > > > Add function > linkend="functions-info-partition">pg_partition_tree() > to display information about partitions (Amit Langote) > > > > Can we combine these three? Good idea, done. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Wed, May 22, 2019 at 04:50:14PM +0900, Ian Barwick wrote: > On 5/22/19 4:26 PM, Michael Paquier wrote: > > On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: > > > the last two items are performance improvements not related to > > > authentication; > > > presumably the VACUUM item would be better off in the "Utility Commands" > > > section and the TRUNCATE item in "General Performance"? > > > > I agree with removing them from authentication, but these are not > > performance-related items. Instead I think that "Utility commands" is > > a place where they can live better. > > > > I am wondering if we should insist on the DOS attacks on a server, as > > non-authorized users are basically able to block any tables, and > > authorization is only a part of it, one of the worst parts > > actually... Anyway, I think that "This prevents unauthorized locking > > delays." does not provide enough details. What about reusing the > > first paragraph of the commits? Here is an idea: > > "A caller of TRUNCATE/VACUUM/ANALYZE could previously queue for an > > access exclusive lock on a relation it may not have permission to > > truncate/vacuum/analyze, potentially interfering with users authorized > > to work on it. This could prevent users from accessing some relations > > they have access to, in some cases preventing authentication if a > > critical catalog relation was blocked." > > Ah, if that's the intent behind/use for those changes (I haven't looked at > them > in any detail, was just scanning the release notes) then it certainly needs > some > explanation along those lines. Since we did not backpatch this fix, I am hesitant to spell out exactly how to exploit this DOS attack. Yes, people can read it in the email archives, and commit messages, but I don't see the value in spelling it out the release notes too. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Tue, May 21, 2019 at 12:57:56PM -0700, Andres Freund wrote: > Hi, > > On 2019-05-21 15:47:34 -0400, Bruce Momjian wrote: > > On Mon, May 20, 2019 at 03:17:19PM -0700, Andres Freund wrote: > > > Hi, > > > > > > Note that I've added a few questions to individuals involved with > > > specific points. If you're in the To: list, please search for your name. > > > > > > > > > On 2019-05-11 16:33:24 -0400, Bruce Momjian wrote: > > > > I have posted a draft copy of the PG 12 release notes here: > > > > > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. > > > > > > Thanks! > > > > > > Migration to Version 12 > > > > > > There's a number of features in the compat section that are more general > > > improvements with a side of incompatibility. Won't it be confusing to > > > e.g. have have the ryu floating point conversion speedups in the compat > > > section, but not in the "General Performance" section? > > > > Yes, it can be. What I did with the btree item was to split out the max > > length change with the larger changes. We can do the same for other > > items. As you rightly stated, it is for cases where the incompatibility > > is minor compared to the change. Do you have a list of the ones that > > need this treatment? > > I was concretely thinking of: > - floating point output changes, which are primarily about performance If we split out the compatibility change, we don't have much left but "performance", and that doesn't seem long enough for an entry. > - recovery.conf changes where I'd merge: > - Do not allow multiple different recovery_target specificatios > - Allow some recovery parameters to be changed with reload > - Cause recovery to advance to the latest timeline by default > - Add an explicit value of current for guc-recovery-target-time > into on entry on the feature side. > > After having to move recovery settings to a different file, disallowing > multiple targets isn't really a separate config break imo. And all the > other changes are also fallout from the recovery.conf GUCification. Even though we moved the recovery.conf values into postgresql.conf, I think people will assume they just behave the same and copy them into the new file. If their behavior changes, they need to know that explicitly. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Tue, May 21, 2019 at 02:22:53PM -0700, Peter Geoghegan wrote:
> On Tue, May 21, 2019 at 1:51 PM Bruce Momjian wrote:
> > > My concern here (which I believe Alexander shares) is that it doesn't
> > > make sense to group these two items together. They're two totally
> > > unrelated pieces of work. Alexander's work does more or less help with
> > > lock contention with writes, whereas the feature that that was merged
> > > with is about preventing index bloat, which is mostly helpful for
> > > reads (it helps writes to the extent that writes are also reads).
> > >
> > > The release notes go on to say that this item "gives better
> > > performance for UPDATEs and DELETEs on indexes with many duplicates",
> > > which is wrong. That is something that should have been listed below,
> > > under the "duplicate index entries in heap-storage order" item.
> >
> > OK, I understand how the lock stuff improves things, but I have
> > forgotten how indexes are made smaller. Is it because of better page
> > split logic?
>
> That is clearly the main reason, though suffix truncation (which
> represents that trailing/suffix columns in index tuples from branch
> pages have "negative infinity" sentinel values) also contributes to
> making indexes smaller.
>
> The page split stuff was mostly added by commit fab250243 ("Consider
> secondary factors during nbtree splits"), but commit f21668f32 ("Add
> "split after new tuple" nbtree optimization") added to that in a way
> that really helped the TPC-C indexes. The TPC-C indexes are about 40%
> smaller now.
First, my apologies in getting to this so late. Peter Geoghegan
supplied me with slides and a video to study, and I now understand how
complex the btree improvements are. Here is a video of Peter's
presentation at PGCon:
https://www.youtube.com/watch?v=p5RaATILoiE
What I would like to do is to type them out here, and if I got it right,
I can then add these details to the release notes.
The over-arching improvement to btree in PG 12 is the ordering of index
entries by tid so all entries are unique. As Peter has stated, many
optimizations come out of that:
1. Since all index tuples are ordered, you can move from one leaf page
to the next without keeping a lock on the internal page that references
it, increasing concurrency.
2. When inserting a duplicate value in the index, we used to try a few
pages to see if there was space, then "get tired" and just split a page
containing duplicates. Now that there are no duplicates (because
duplicate key fields are sorted by tid) the system knows exactly what
page the index tuple belongs on, and inserts or splits the page as
necessary.
3. Pivot tuples are used on internal pages and as min/max indicators on
leaf pages. These pivot tuples are now trimmed if their trailing key
fields are not significant. For example, if an index is
field1/field2/field3, and the page contains values for which field1==5
and none that field1==6, there is no need to include field2 and field3
in the pivot tuple --- it can just list the pivot as field1==5,
field2=infinity. This is called suffix truncation.
Page splits used to just split the page in half, which minimizes the
number of page splits, but sometimes causes lots of wasted space. The
new code tries to split to reduce the length of pivot tuples, which ends
up being more efficient in space savings because the pivot tuples are
shorter, and the leaf pages end up being more tightly packed. This is
particularly true for ever-increasing keys because we often end up
creating a new empty page, rather than splitting an existing page.
4. Vacuum's removal of index tuples in indexes with many duplicates is
faster since it can more quickly find desired tuples.
Did I cover everything?
--
Bruce Momjian http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
Re: PG 12 draft release notes
On 5/11/19 10:33 PM, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. > Hello, By looking to a user request to add greek in our FTS [1], I suggest to mention which languages has been added in fd582317e. Patch attached. I hesitate to also mention these changes? > These all work in UTF8, and the indonesian and irish ones also work in LATIN1. > The non-UTF8 version of the hungarian stemmer now works in LATIN2 not LATIN1. 1: https://www.postgresql.org/message-id/trinity-f09793a1-8c13-4b56-94fe-10779e96c87e-1559896268438%403c-app-mailcom-bs16 Cheers, -- Adrien diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml index 532055456c..c50a23bd1c 100644 --- a/doc/src/sgml/release-12.sgml +++ b/doc/src/sgml/release-12.sgml @@ -2085,8 +2085,8 @@ Author: Tom Lane --> - Update Snowball stemmer dictionaries with support for new languages - (Arthur Zakirov) + Update Snowball stemmer dictionaries with support for new languages: + arabic, indonesian, irish, lithuanian, nepali and tami. (Arthur Zakirov) signature.asc Description: OpenPGP digital signature
Re: PG 12 draft release notes
On 2019/05/29 0:58, Andres Freund wrote: > Hi, > > > > > > Add function > linkend="functions-info-partition">pg_partition_root() > to return top-most parent of a partition tree (Michaël Paquier) > > > > > > > > Add function > linkend="functions-info-partition">pg_partition_ancestors() > to report all ancestors of a partition (Álvaro Herrera) > > > > > > > > Add function > linkend="functions-info-partition">pg_partition_tree() > to display information about partitions (Amit Langote) > > > > Can we combine these three? +1 Thanks, Amit
Re: PG 12 draft release notes
Hi, Add function pg_partition_root() to return top-most parent of a partition tree (Michaël Paquier) Add function pg_partition_ancestors() to report all ancestors of a partition (Álvaro Herrera) Add function pg_partition_tree() to display information about partitions (Amit Langote) Can we combine these three? Greetings, Andres Freund
Re: PG 12 draft release notes
On Tue, May 21, 2019 at 05:00:31PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Mon, May 20, 2019 at 08:48:15PM -0400, Tom Lane wrote: > >> Yes, this should be in "source code". I think it should be merged > >> with a391ff3c and 74dfe58a into something like > >> > >> Allow extensions to create planner support functions that > >> can provide function-specific selectivity, cost, and > >> row-count estimates that can depend on the function arguments. > >> Support functions can also transform WHERE clauses involving > >> an extension's functions and operators into indexable clauses > >> in ways that the core code cannot for lack of detailed semantic > >> knowledge of those functions/operators. > > > The new text is: > > > Add support function capability to improve optimizer estimates > > for functions (Tom Lane) > > > This allows extensions to create planner support functions that > > can provide function-specific selectivity, cost, and row-count > > estimates that can depend on the function arguments. Also, improve > > in-core estimates for generate_series(), > > unnest(), and functions that return boolean > > values. > > Uh ... you completely lost the business about custom indexable clauses. > I agree with Andres that that's the most important aspect of this. Oh, I see what you mean now. I have updated the docs and moved the item to Source Code: Add support function capability to improve optimizer estimates, inlining, and indexing for functions (Tom Lane) This allows extensions to create planner support functions that can provide function-specific selectivity, cost, and row-count estimates that can depend on the function's arguments. Support functions can also supply simplified representations and index conditions, greatly expanding optimization possibilities. > > Notice that there are some improvments in in-core functions. Should this > > still be moved to the source code section? > > I doubt that that's worth mentioning at all. It certainly isn't a > reason not to move this to the source-code section, because that's > where we generally put things that are of interest for improving > extensions, which is what this mainly is. In-core function mention removed. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On 5/22/19 4:26 PM, Michael Paquier wrote: On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: the last two items are performance improvements not related to authentication; presumably the VACUUM item would be better off in the "Utility Commands" section and the TRUNCATE item in "General Performance"? I agree with removing them from authentication, but these are not performance-related items. Instead I think that "Utility commands" is a place where they can live better. I am wondering if we should insist on the DOS attacks on a server, as non-authorized users are basically able to block any tables, and authorization is only a part of it, one of the worst parts actually... Anyway, I think that "This prevents unauthorized locking delays." does not provide enough details. What about reusing the first paragraph of the commits? Here is an idea: "A caller of TRUNCATE/VACUUM/ANALYZE could previously queue for an access exclusive lock on a relation it may not have permission to truncate/vacuum/analyze, potentially interfering with users authorized to work on it. This could prevent users from accessing some relations they have access to, in some cases preventing authentication if a critical catalog relation was blocked." Ah, if that's the intent behind/use for those changes (I haven't looked at them in any detail, was just scanning the release notes) then it certainly needs some explanation along those lines. Regards Ian Barwick -- Ian Barwick https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Re: PG 12 draft release notes
On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: > the last two items are performance improvements not related to authentication; > presumably the VACUUM item would be better off in the "Utility Commands" > section and the TRUNCATE item in "General Performance"? I agree with removing them from authentication, but these are not performance-related items. Instead I think that "Utility commands" is a place where they can live better. I am wondering if we should insist on the DOS attacks on a server, as non-authorized users are basically able to block any tables, and authorization is only a part of it, one of the worst parts actually... Anyway, I think that "This prevents unauthorized locking delays." does not provide enough details. What about reusing the first paragraph of the commits? Here is an idea: "A caller of TRUNCATE/VACUUM/ANALYZE could previously queue for an access exclusive lock on a relation it may not have permission to truncate/vacuum/analyze, potentially interfering with users authorized to work on it. This could prevent users from accessing some relations they have access to, in some cases preventing authentication if a critical catalog relation was blocked." -- Michael signature.asc Description: PGP signature
Re: PG 12 draft release notes
On Tue, May 21, 2019 at 3:49 AM Peter Geoghegan wrote: > On Mon, May 20, 2019 at 3:17 PM Andres Freund wrote: > > > > > > > > Improve speed of btree index insertions (Peter Geoghegan, > > Alexander Korotkov) > > > > My concern here (which I believe Alexander shares) is that it doesn't > make sense to group these two items together. They're two totally > unrelated pieces of work. Alexander's work does more or less help with > lock contention with writes, whereas the feature that that was merged > with is about preventing index bloat, which is mostly helpful for > reads (it helps writes to the extent that writes are also reads). > > The release notes go on to say that this item "gives better > performance for UPDATEs and DELETEs on indexes with many duplicates", > which is wrong. That is something that should have been listed below, > under the "duplicate index entries in heap-storage order" item. +1 -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Re: PG 12 draft release notes
On 5/12/19 5:33 AM, Bruce Momjian wrote: I have posted a draft copy of the PG 12 release notes here: http://momjian.us/pgsql_docs/release-12.html They are committed to git. It includes links to the main docs, where appropriate. Our official developer docs will rebuild in a few hours. In section "Authentication": https://www.postgresql.org/docs/devel/release-12.html#id-1.11.6.5.5.3.7 the last two items are performance improvements not related to authentication; presumably the VACUUM item would be better off in the "Utility Commands" section and the TRUNCATE item in "General Performance"? In section "Source code": https://www.postgresql.org/docs/devel/release-12.html#id-1.11.6.5.5.12 the item "Add CREATE ACCESS METHOD command" doesn't seem related to the source code itself, though I'm not sure where it should go. Regards Ian Barwick -- Ian Barwick https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Re: PG 12 draft release notes
On Tue, May 21, 2019 at 1:51 PM Bruce Momjian wrote:
> > My concern here (which I believe Alexander shares) is that it doesn't
> > make sense to group these two items together. They're two totally
> > unrelated pieces of work. Alexander's work does more or less help with
> > lock contention with writes, whereas the feature that that was merged
> > with is about preventing index bloat, which is mostly helpful for
> > reads (it helps writes to the extent that writes are also reads).
> >
> > The release notes go on to say that this item "gives better
> > performance for UPDATEs and DELETEs on indexes with many duplicates",
> > which is wrong. That is something that should have been listed below,
> > under the "duplicate index entries in heap-storage order" item.
>
> OK, I understand how the lock stuff improves things, but I have
> forgotten how indexes are made smaller. Is it because of better page
> split logic?
That is clearly the main reason, though suffix truncation (which
represents that trailing/suffix columns in index tuples from branch
pages have "negative infinity" sentinel values) also contributes to
making indexes smaller.
The page split stuff was mostly added by commit fab250243 ("Consider
secondary factors during nbtree splits"), but commit f21668f32 ("Add
"split after new tuple" nbtree optimization") added to that in a way
that really helped the TPC-C indexes. The TPC-C indexes are about 40%
smaller now.
> > > Author: Peter Geoghegan
> > > 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column.
> As I remember the benefit currently is that you can find update and
> deleted rows faster, right?
Yes, that's true when writing to the index. But more importantly, it
really helps VACUUM when there are lots of duplicates, which is fairly
common in the real world (imagine an index where 20% of the rows are
NULL, for example). In effect, there are no duplicates anymore,
because all index tuples are unique internally.
Indexes with lots of duplicates group older rows together, and new
rows together, because treating heap TID as a tiebreaker naturally has
that effect. VACUUM will generally dirty far fewer pages, because bulk
deletions tend to be correlated with heap TID. And, VACUUM has a much
better chance of deleting entire leaf pages, because dead tuples end
up getting grouped together.
--
Peter Geoghegan
Re: PG 12 draft release notes
Bruce Momjian writes: > On Mon, May 20, 2019 at 08:48:15PM -0400, Tom Lane wrote: >> Yes, this should be in "source code". I think it should be merged >> with a391ff3c and 74dfe58a into something like >> >> Allow extensions to create planner support functions that >> can provide function-specific selectivity, cost, and >> row-count estimates that can depend on the function arguments. >> Support functions can also transform WHERE clauses involving >> an extension's functions and operators into indexable clauses >> in ways that the core code cannot for lack of detailed semantic >> knowledge of those functions/operators. > The new text is: > Add support function capability to improve optimizer estimates > for functions (Tom Lane) > This allows extensions to create planner support functions that > can provide function-specific selectivity, cost, and row-count > estimates that can depend on the function arguments. Also, improve > in-core estimates for generate_series(), > unnest(), and functions that return boolean > values. Uh ... you completely lost the business about custom indexable clauses. I agree with Andres that that's the most important aspect of this. > Notice that there are some improvments in in-core functions. Should this > still be moved to the source code section? I doubt that that's worth mentioning at all. It certainly isn't a reason not to move this to the source-code section, because that's where we generally put things that are of interest for improving extensions, which is what this mainly is. regards, tom lane
Re: PG 12 draft release notes
On Mon, May 20, 2019 at 05:48:50PM -0700, Peter Geoghegan wrote: > On Mon, May 20, 2019 at 3:17 PM Andres Freund wrote: > > > > > > > > Improve speed of btree index insertions (Peter Geoghegan, > > Alexander Korotkov) > > > > My concern here (which I believe Alexander shares) is that it doesn't > make sense to group these two items together. They're two totally > unrelated pieces of work. Alexander's work does more or less help with > lock contention with writes, whereas the feature that that was merged > with is about preventing index bloat, which is mostly helpful for > reads (it helps writes to the extent that writes are also reads). > > The release notes go on to say that this item "gives better > performance for UPDATEs and DELETEs on indexes with many duplicates", > which is wrong. That is something that should have been listed below, > under the "duplicate index entries in heap-storage order" item. OK, I understand how the lock stuff improves things, but I have forgotten how indexes are made smaller. Is it because of better page split logic? > > Author: Peter Geoghegan > > 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column. > > Author: Peter Geoghegan > > 2019-03-20 [fab250243] Consider secondary factors during nbtree splits. > > --> > > > > > > Have new btree indexes sort duplicate index entries in heap-storage > > order (Peter Geoghegan, Heikki Linnakangas) > > > > > I'm not sure that the grouping here is quite right. And the second entry > > probably should have some explanation about the benefits? > > It could stand to say something about the benefits. As I said, there > is already a little bit about the benefits, but that ended up being > tied to the "Improve speed of btree index insertions" item. Moving > that snippet to the correct item would be a good start. As I remember the benefit currently is that you can find update and deleted rows faster, right? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Mon, May 20, 2019 at 08:48:15PM -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-05-20 18:56:50 -0400, Tom Lane wrote: > >> I'm not sure which of my commits you want me to opine on, other than > > > That was one of the main ones. I'm also specifically wondering about: > > >> Author: Tom Lane > >> 2019-02-09 [1fb57af92] Create the infrastructure for planner support > >> functions. > >> > >> Add support for function > >> selectivity (Tom Lane) > >> > >> > >> > >> Hm, that message doesn't seem like an accurate description of that > >> commit (if anything it's a391ff3c?). Given that it all requires C > >> hackery, perhaps we ought to move it to the source code section? > > Yes, this should be in "source code". I think it should be merged > with a391ff3c and 74dfe58a into something like > > Allow extensions to create planner support functions that > can provide function-specific selectivity, cost, and > row-count estimates that can depend on the function arguments. > Support functions can also transform WHERE clauses involving > an extension's functions and operators into indexable clauses > in ways that the core code cannot for lack of detailed semantic > knowledge of those functions/operators. The new text is: Add support function capability to improve optimizer estimates for functions (Tom Lane) This allows extensions to create planner support functions that can provide function-specific selectivity, cost, and row-count estimates that can depend on the function arguments. Also, improve in-core estimates for generate_series(), unnest(), and functions that return boolean values. Notice that there are some improvments in in-core functions. Should this still be moved to the source code section? > > and perhaps you could opine on whether we ought to include > > >> > >> > >> > >> > >> Improve handling of partition dependency (Tom Lane) > >> > >> > >> > >> This prevents the creation of inconsistent partition hierarchies > >> in rare cases. > >> > >> > > It's probably worth mentioning, but I'd say something like > > Fix bugs that could cause ALTER TABLE DETACH PARTITION > to not drop objects that should be dropped, such as > automatically-created child indexes. > > The rest of it is not terribly interesting from a user's standpoint, > I think. Done. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Mon, May 20, 2019 at 06:56:50PM -0400, Tom Lane wrote: > Andres Freund writes: > > Note that I've added a few questions to individuals involved with > > specific points. If you're in the To: list, please search for your name. > > I'm not sure which of my commits you want me to opine on, other than > > > > > > > > >Allow RECORD and RECORD[] to be specified > >as a function return-value > >record (Elvis Pranskevichus) > > > > > >DETAIL? > > > > > > > This description doesn't sound accurate to me. Tom? > > Yeah, maybe better > >Allow RECORD and RECORD[] to be used >as column types in a query's column definition list for a >table function that is declared to return RECORD >(Elvis Pranskevichus) > > You could link to "queries-tablefunctions" which describes the column > definition business; it's much more specific than "sql-createfunction". Done. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Tue, May 21, 2019 at 12:04:50PM +1200, David Rowley wrote: > On Tue, 21 May 2019 at 10:17, Andres Freund wrote: > > commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1 > > Author: Andres Freund > > Date: 2018-11-16 16:35:11 -0800 > > > > Make TupleTableSlots extensible, finish split of existing slot type. > > > >Given that those commits entail an API break relevant for extensions, > >should we have them as a separate "source code" note? > > > > 4) I think the attribution isn't quite right. For one, a few names with > >substantial work are missing (Amit Khandekar, Ashutosh Bapat, > >Alexander Korotkov), and the order doesn't quite seem right. On the > >latter part I might be somewhat petty, but I spend *many* months of > >my life on this. > > > >How about: > >Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, David > > Rowley, Dimitri Golgov > >if we keep 3) separate and > >Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, Alexander > > Korotkov, Amit Khandekar, David Rowley, Dimitri Golgov > >otherwise? > > > >I think it might actually make sense to take David off this list, > >because his tableam work is essentially part of it's own entry, as > > Yeah, please take me off that one. My focus there was mostly on > keeping COPY fast with partitioned tables, to which, as Andres > mentioned is listed somewhere else. Done. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
Hi, On 2019-05-21 15:47:34 -0400, Bruce Momjian wrote: > On Mon, May 20, 2019 at 03:17:19PM -0700, Andres Freund wrote: > > Hi, > > > > Note that I've added a few questions to individuals involved with > > specific points. If you're in the To: list, please search for your name. > > > > > > On 2019-05-11 16:33:24 -0400, Bruce Momjian wrote: > > > I have posted a draft copy of the PG 12 release notes here: > > > > > > http://momjian.us/pgsql_docs/release-12.html > > > They are committed to git. > > > > Thanks! > > > > Migration to Version 12 > > > > There's a number of features in the compat section that are more general > > improvements with a side of incompatibility. Won't it be confusing to > > e.g. have have the ryu floating point conversion speedups in the compat > > section, but not in the "General Performance" section? > > Yes, it can be. What I did with the btree item was to split out the max > length change with the larger changes. We can do the same for other > items. As you rightly stated, it is for cases where the incompatibility > is minor compared to the change. Do you have a list of the ones that > need this treatment? I was concretely thinking of: - floating point output changes, which are primarily about performance - recovery.conf changes where I'd merge: - Do not allow multiple different recovery_target specificatios - Allow some recovery parameters to be changed with reload - Cause recovery to advance to the latest timeline by default - Add an explicit value of current for guc-recovery-target-time into on entry on the feature side. After having to move recovery settings to a different file, disallowing multiple targets isn't really a separate config break imo. And all the other changes are also fallout from the recovery.conf GUCification. > > > > Remove the special behavior of > linkend="datatype-oid">OID columns (Andres Freund, > > John Naylor) > > > > > > Should we mention that tables with OIDs have to have their oids removed > > before they can be upgraded? > > Uh, is that true? pg_upgrade? pg_dump? Yes. > > > > Refactor geometric > > functions and operators (Emre Hasegeli) > > > > > > > > This could lead to more accurate, but slightly different, results > > from previous releases. > > > > > > > > > > > > > > Restructure geometric > > types to handle NaN, underflow, overflow and division by > > zero more consistently (Emre Hasegeli) > > > > > > > > > > > > > > > > Improve behavior and error reporting for the > linkend="datatype-geometric">line data type (Emre Hasegeli) > > > > > > > > Is that sufficient explanation? Feels like we need to expand a bit > > more. In particular, is it possible that a subset of the changes here > > require reindexing? > > > > Also, aren't three different entries a bit too much? > > The 'line' item related to more errors than just the ones listed for the > geometric data types, so I was not clear how to do that as a single > entry. I think there is a much larger compatibility breakage > possibility with 'line'. > > > > > > > > > > > Improve speed of btree index insertions (Peter Geoghegan, > > Alexander Korotkov) > > > > > > > > The new code improves the space-efficiency of page splits, > > reduces locking overhead, and gives better performance for > > UPDATEs and DELETEs on > > indexes with many duplicates. > > > > > > > > > > > > > > > > Have new btree indexes sort duplicate index entries in heap-storage > > order (Peter Geoghegan, Heikki Linnakangas) > > > > > > > > Indexes pg_upgraded from previous > > releases will not have this ordering. > > > > > > > > I'm not sure that the grouping here is quite right. And the second entry > > probably should have some explanation about the benefits? > > Agreed. > > > > > > > > > > > Reduce locking requirements for index renaming (Peter Eisentraut) > > > > > > > > Should we specify the newly required lock level? Because it's quire > > relevant for users what exactly they're now able to do concurrently in > > operation? > > Sure. > > > > > > > > > > > Add support for function > > selectivity (Tom Lane) > > > > > > > > Hm, that message doesn't seem like an accurate description of that > > commit (if anything it's a391ff3c?). Given that it all requires C > > hackery, perhaps we ought to move it to the source code section? And > > isn't the most important part of this set of changes > > > > commit 74dfe58a5927b22c744b29534e67bfdd203ac028 > > Author: Tom Lane > > Date: 2019-02-11 21:26:08 -0500 > > > > Allow extensions to generate lossy
Re: PG 12 draft release notes
qOn Mon, May 20, 2019 at 03:17:19PM -0700, Andres Freund wrote: > Hi, > > Note that I've added a few questions to individuals involved with > specific points. If you're in the To: list, please search for your name. > > > On 2019-05-11 16:33:24 -0400, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. > > Thanks! > > Migration to Version 12 > > There's a number of features in the compat section that are more general > improvements with a side of incompatibility. Won't it be confusing to > e.g. have have the ryu floating point conversion speedups in the compat > section, but not in the "General Performance" section? Yes, it can be. What I did with the btree item was to split out the max length change with the larger changes. We can do the same for other items. As you rightly stated, it is for cases where the incompatibility is minor compared to the change. Do you have a list of the ones that need this treatment? > > Remove the special behavior oflinkend="datatype-oid">OID columns (Andres Freund, > John Naylor) > > > Should we mention that tables with OIDs have to have their oids removed > before they can be upgraded? Uh, is that true? pg_upgrade? pg_dump? > > Refactor geometric > functions and operators (Emre Hasegeli) > > > > This could lead to more accurate, but slightly different, results > from previous releases. > > > > > > > Restructure geometric > types to handle NaN, underflow, overflow and division by > zero more consistently (Emre Hasegeli) > > > > > > > > Improve behavior and error reporting for thelinkend="datatype-geometric">line data type (Emre Hasegeli) > > > > Is that sufficient explanation? Feels like we need to expand a bit > more. In particular, is it possible that a subset of the changes here > require reindexing? > > Also, aren't three different entries a bit too much? The 'line' item related to more errors than just the ones listed for the geometric data types, so I was not clear how to do that as a single entry. I think there is a much larger compatibility breakage possibility with 'line'. > > > > > Improve speed of btree index insertions (Peter Geoghegan, > Alexander Korotkov) > > > > The new code improves the space-efficiency of page splits, > reduces locking overhead, and gives better performance for > UPDATEs and DELETEs on > indexes with many duplicates. > > > > > > > > Have new btree indexes sort duplicate index entries in heap-storage > order (Peter Geoghegan, Heikki Linnakangas) > > > > Indexes pg_upgraded from previous > releases will not have this ordering. > > > > I'm not sure that the grouping here is quite right. And the second entry > probably should have some explanation about the benefits? Agreed. > > > > > Reduce locking requirements for index renaming (Peter Eisentraut) > > > > Should we specify the newly required lock level? Because it's quire > relevant for users what exactly they're now able to do concurrently in > operation? Sure. > > > > > Add support for function > selectivity (Tom Lane) > > > > Hm, that message doesn't seem like an accurate description of that > commit (if anything it's a391ff3c?). Given that it all requires C > hackery, perhaps we ought to move it to the source code section? And > isn't the most important part of this set of changes > > commit 74dfe58a5927b22c744b29534e67bfdd203ac028 > Author: Tom Lane > Date: 2019-02-11 21:26:08 -0500 > > Allow extensions to generate lossy index conditions. Uh, I missed that as an important item. Can someone give me some text? > > > > > Greatly reduce memory consumption of > and function calls (Andres Freund, Tomas Vondra, Tom Lane) > > > > Grouping these three changes together makes no sense to me. > > I think the first commit just ought not to be mentioned separately, it's > just a fix for a memory leak in 31f3817402, essentially a 12 only bugfix? Oh, I was not aware of that. > The second commit is about position() etc, which seems not to match that > description either? Ugh. > The third is probably more appropriate to be in the source code > section. While it does speed up function calls a bit (in particular > plpgsql which is very function call heavy), it also is a breaking change > for some external code? Not sure why Tom is listed with this entry? The order of names is just a guess when multiple commits are merged --- this needs help. > > > >
Re: PG 12 draft release notes
On Tue, May 21, 2019 at 09:09:10AM -0700, Shawn Debnath wrote: > On Sat, May 11, 2019 at 04:33:24PM -0400, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main docs, where > > appropriate. Our official developer docs will rebuild in a few hours. > > > > Thank you for doing this. I didn't see [1] in the release notes, should > it be included in the "Source Code" section? > > [1] > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=3eb77eba5a51780d5cf52cd66a9844cd4d26feb0 Uh, this is an internals change that is usually not listed in the release notes since it mostly affects internals developers. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On 2019-05-21 00:17, Andres Freund wrote: > > > > > Reduce locking requirements for index renaming (Peter Eisentraut) > > > > Should we specify the newly required lock level? Because it's quire > relevant for users what exactly they're now able to do concurrently in > operation? Yes, more information is in the commit message. We could expand the release note item with: """ Renaming an index now requires only a ShareUpdateExclusiveLock instead of a AccessExclusiveLock. This allows index renaming without blocking access to the table. """ Note also that this functionality later became part of REINDEX CONCURRENTLY, which is presumably where most people will make use of it. > > > > >Add an explicit value of current for linkend="guc-recovery-target-time"/> (Peter Eisentraut) > > > > Seems like this should be combined with the earlier "Cause recovery to > advance to the latest timeline by default" entry. It could be combined or kept separate or not mentioned at all. Either way is fine. > > > > >Add support for generated >columns (Peter Eisentraut) > > > >Rather than storing a value only at row creation time, generated >columns are also modified during updates, and can reference other >table columns. > > > > I find this description confusing. How about cribbing from the commit? > Roughly like > > This allows creating columns that are computed from expressions, > including references to other columns in the same table, rather than > having to be specified by the inserter/updater. Yeah, that's better. > Think we also ought to mention that this is only stored generated > columns, given that the SQL feature also includes virtual columns? The SQL standard doesn't specify whether generated columns are stored, but reading between the lines suggest that they expect them to be. So we don't need to get into more detail there in the release notes. The main documentation does address this point. > > > > >Allow modifications of system tables using linkend="sql-altertable"/> (Peter Eisentraut) > > > >This allows modifications of reloptions and >autovacuum settings. > > > > I think the first paragraph is a bit dangerous. This does *not* > generally allow modifications of system tables using ALTER TABLE. Yes, it's overly broad. The second paragraph is really the gist of the change, so we could write Allow modifications of reloptions of system tables -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 12 draft release notes
On Tue, May 21, 2019 at 8:17 AM Andres Freund wrote: > > >Add command to create >new table types (Haribabu Kommi, Andres Freund, Álvaro Herrera, >Dimitri Dolgov) > > > A few points: > > 1) Is this really source code, given that CREATE ACCESS METHOD TYPE >TABLE is a DDL command, and USING (...) for CREATE TABLE etc is an >option to DDL commands? > +1 It would be better to provide a description of the newly added syntax. Do we need to provide any 'Note' explaining that currently there are no other alternatives to the heap? 2) I think the description sounds a bit too much like it's about new >forms of tables, rather than their storage. How about something >roughly like: > >Allow different table access methods to be >used. This allows to develop and >use new ways of storing and accessing table data, optimized for >different use-cases, without having to modify >PostgreSQL. The existing heap access method >remains the default. > > 3) This misses a large set of commits around making tableam possible, in >particular the commits around > > commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1 > Author: Andres Freund > Date: 2018-11-16 16:35:11 -0800 > > Make TupleTableSlots extensible, finish split of existing slot type. > >Given that those commits entail an API break relevant for extensions, >should we have them as a separate "source code" note? > +1 to add, but I am not sure whether we need to list all the breakage that has introduced by Tableam needs to be described separately or with some combined note to explain it to extension developers is fine? > 4) I think the attribution isn't quite right. For one, a few names with >substantial work are missing (Amit Khandekar, Ashutosh Bapat, >Alexander Korotkov), and the order doesn't quite seem right. On the >latter part I might be somewhat petty, but I spend *many* months of >my life on this. > >How about: >Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, > David Rowley, Dimitri Golgov >if we keep 3) separate and >Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, > Alexander Korotkov, Amit Khandekar, David Rowley, Dimitri Golgov >otherwise? +1 to either of the above. Without Andres enormous efforts, Tableam couldn't have been possible into v12. Regards, Haribabu Kommi Fujitsu Australia
Re: PG 12 draft release notes
On Mon, May 20, 2019 at 3:17 PM Andres Freund wrote: > > > > Improve speed of btree index insertions (Peter Geoghegan, > Alexander Korotkov) > My concern here (which I believe Alexander shares) is that it doesn't make sense to group these two items together. They're two totally unrelated pieces of work. Alexander's work does more or less help with lock contention with writes, whereas the feature that that was merged with is about preventing index bloat, which is mostly helpful for reads (it helps writes to the extent that writes are also reads). The release notes go on to say that this item "gives better performance for UPDATEs and DELETEs on indexes with many duplicates", which is wrong. That is something that should have been listed below, under the "duplicate index entries in heap-storage order" item. > Author: Peter Geoghegan > 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column. > Author: Peter Geoghegan > 2019-03-20 [fab250243] Consider secondary factors during nbtree splits. > --> > > > Have new btree indexes sort duplicate index entries in heap-storage > order (Peter Geoghegan, Heikki Linnakangas) > > I'm not sure that the grouping here is quite right. And the second entry > probably should have some explanation about the benefits? It could stand to say something about the benefits. As I said, there is already a little bit about the benefits, but that ended up being tied to the "Improve speed of btree index insertions" item. Moving that snippet to the correct item would be a good start. -- Peter Geoghegan
Re: PG 12 draft release notes
Andres Freund writes: > On 2019-05-20 18:56:50 -0400, Tom Lane wrote: >> I'm not sure which of my commits you want me to opine on, other than > That was one of the main ones. I'm also specifically wondering about: >> Author: Tom Lane >> 2019-02-09 [1fb57af92] Create the infrastructure for planner support >> functions. >> >> Add support for function >> selectivity (Tom Lane) >> >> >> >> Hm, that message doesn't seem like an accurate description of that >> commit (if anything it's a391ff3c?). Given that it all requires C >> hackery, perhaps we ought to move it to the source code section? Yes, this should be in "source code". I think it should be merged with a391ff3c and 74dfe58a into something like Allow extensions to create planner support functions that can provide function-specific selectivity, cost, and row-count estimates that can depend on the function arguments. Support functions can also transform WHERE clauses involving an extension's functions and operators into indexable clauses in ways that the core code cannot for lack of detailed semantic knowledge of those functions/operators. > and perhaps you could opine on whether we ought to include >> >> >> >> >> Improve handling of partition dependency (Tom Lane) >> >> >> >> This prevents the creation of inconsistent partition hierarchies >> in rare cases. >> >> It's probably worth mentioning, but I'd say something like Fix bugs that could cause ALTER TABLE DETACH PARTITION to not drop objects that should be dropped, such as automatically-created child indexes. The rest of it is not terribly interesting from a user's standpoint, I think. > And lastly, opine on the int GUC fractions, microsoecond, and cost_delay > items? I agree with your comments on those. regards, tom lane
Re: PG 12 draft release notes
On Tue, 21 May 2019 at 10:17, Andres Freund wrote: > commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1 > Author: Andres Freund > Date: 2018-11-16 16:35:11 -0800 > > Make TupleTableSlots extensible, finish split of existing slot type. > >Given that those commits entail an API break relevant for extensions, >should we have them as a separate "source code" note? > > 4) I think the attribution isn't quite right. For one, a few names with >substantial work are missing (Amit Khandekar, Ashutosh Bapat, >Alexander Korotkov), and the order doesn't quite seem right. On the >latter part I might be somewhat petty, but I spend *many* months of >my life on this. > >How about: >Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, David > Rowley, Dimitri Golgov >if we keep 3) separate and >Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, Alexander > Korotkov, Amit Khandekar, David Rowley, Dimitri Golgov >otherwise? > >I think it might actually make sense to take David off this list, >because his tableam work is essentially part of it's own entry, as Yeah, please take me off that one. My focus there was mostly on keeping COPY fast with partitioned tables, to which, as Andres mentioned is listed somewhere else. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Re: PG 12 draft release notes
Hi, On 2019-05-21 00:08:25 +0100, Andrew Gierth wrote: > > "Andres" == Andres Freund writes: > > Andres> Any chance for you to propose a text? > > This is what I posted before; I'm not 100% happy with it but it's still > better than any of the other versions: > * Output REAL and DOUBLE PRECISION values in shortest-exact format by >default, and change the behavior of extra_float_digits > >Previously, float values were output rounded to 6 or 15 decimals by >default, with the number of decimals adjusted by extra_float_digits. >The previous rounding behavior is no longer the default, and is now >done only if extra_float_digits is set to zero or less; if the value >is greater than zero (which it is by default), a shortest-precise >representation is output (for a substantial performance improvement). >This representation preserves the exact binary value when correctly >read back in, even though the trailing digits will usually differ >from the output generated by previous versions when >extra_float_digits=3. Definitely better from what's there in my opinion. Shortening it if reasonable wouldn't hurt. Perhaps Output REAL and DOUBLE PRECISION values in shortest-exact format by default, and change the behavior of extra_float_digits (...) When extra_float_digits is set to a value greater than zero (the default), a shortest-precise representation is output (for a substantial performance improvement). This representation preserves the exact binary value when correctly read back in, even though the trailing digits will usually differ from the output generated by previous versions when extra_float_digits=3. Previously, float values were output rounded to 6 or 15 decimals by default, with the number of decimals adjusted by extra_float_digits. This behaviour can be restored by setting extra_float_digits is set to zero or less. Or something in that vein? Greetings, Andres Freund
Re: PG 12 draft release notes
On 2019-05-20 18:56:50 -0400, Tom Lane wrote: > Andres Freund writes: > > Note that I've added a few questions to individuals involved with > > specific points. If you're in the To: list, please search for your name. > > I'm not sure which of my commits you want me to opine on, other than That was one of the main ones. I'm also specifically wondering about: > > > > > Add support for function > selectivity (Tom Lane) > > > > Hm, that message doesn't seem like an accurate description of that > commit (if anything it's a391ff3c?). Given that it all requires C > hackery, perhaps we ought to move it to the source code section? And > isn't the most important part of this set of changes > > commit 74dfe58a5927b22c744b29534e67bfdd203ac028 > Author: Tom Lane > Date: 2019-02-11 21:26:08 -0500 > > Allow extensions to generate lossy index conditions. and perhaps you could opine on whether we ought to include > > > > > Improve handling of partition dependency (Tom Lane) > > > > This prevents the creation of inconsistent partition hierarchies > in rare cases. > > > > That seems not very informative for users? and if so provide a better description? Because no user is going to make sense of that. And lastly, opine on the int GUC fractions, microsoecond, and cost_delay items? > Yeah, maybe better > >Allow RECORD and RECORD[] to be used >as column types in a query's column definition list for a >table function that is declared to return RECORD >(Elvis Pranskevichus) > > You could link to "queries-tablefunctions" which describes the column > definition business; it's much more specific than "sql-createfunction". Yea, that's much better. Greetings, Andres Freund
Re: PG 12 draft release notes
> "Andres" == Andres Freund writes: Andres> Any chance for you to propose a text? This is what I posted before; I'm not 100% happy with it but it's still better than any of the other versions: * Output REAL and DOUBLE PRECISION values in shortest-exact format by default, and change the behavior of extra_float_digits Previously, float values were output rounded to 6 or 15 decimals by default, with the number of decimals adjusted by extra_float_digits. The previous rounding behavior is no longer the default, and is now done only if extra_float_digits is set to zero or less; if the value is greater than zero (which it is by default), a shortest-precise representation is output (for a substantial performance improvement). This representation preserves the exact binary value when correctly read back in, even though the trailing digits will usually differ from the output generated by previous versions when extra_float_digits=3. -- Andrew (irc:RhodiumToad)
Re: PG 12 draft release notes
Hi, On 2019-05-20 23:56:33 +0100, Andrew Gierth wrote: > [ To: header pruned ] > > > "Andres" == Andres Freund writes: > > Andres> > Andres> Avoid performing unnecessary rounding of Andres> linkend="datatype-float">REAL and > DOUBLE > Andres> PRECISION values (Andrew Gierth) > Andres> > > Andres> > Andres> This dramatically speeds up processing of floating-point > Andres> values but causes additional trailing digits to > Andres> potentially be displayed. Users wishing to have output > Andres> that is rounded to match the previous behavior can set Andres> > linkend="guc-extra-float-digits">extra_float_digits=0, > Andres> which is no longer the default. > Andres> > Andres> > > Andres> Isn't it exactly the *other* way round? *Previously* we'd > Andres> output additional trailing digits. The new algorithm instead > Andres> will instead have *exactly* the required number of digits? > > Yeah, this wording is not right. But your description is also wrong. > > Previously we output values rounded to 6+d or 15+d digits where > d=extra_float_digits, with d=0 being the default. Only clients that > wanted exact results would set that to 3 instead. > > Now we output the minimum digits to get an exact result, which is > usually 8 or 17 digits (sometimes less depending on the value, or 9 for > the relatively rare float4 values that need it) for any > extra_float_digits value > 0. Clients that set d=3 will therefore > usually get one less digit than before, and the value they get will > usually be slightly different (i.e. not the same value that they would > have seen with d=2), but it should give them the same binary value after > going through strtod() or strtof(). Any chance for you to propose a text? Greetings, Andres Freund
Re: PG 12 draft release notes
Andres Freund writes: > Note that I've added a few questions to individuals involved with > specific points. If you're in the To: list, please search for your name. I'm not sure which of my commits you want me to opine on, other than > > > >Allow RECORD and RECORD[] to be specified >as a function return-value >record (Elvis Pranskevichus) > > >DETAIL? > > > This description doesn't sound accurate to me. Tom? Yeah, maybe better Allow RECORD and RECORD[] to be used as column types in a query's column definition list for a table function that is declared to return RECORD (Elvis Pranskevichus) You could link to "queries-tablefunctions" which describes the column definition business; it's much more specific than "sql-createfunction". regards, tom lane
Re: PG 12 draft release notes
[ To: header pruned ] > "Andres" == Andres Freund writes: Andres> Andres> Avoid performing unnecessary rounding oflinkend="datatype-float">REAL and DOUBLE Andres> PRECISION values (Andrew Gierth) Andres> Andres> Andres> This dramatically speeds up processing of floating-point Andres> values but causes additional trailing digits to Andres> potentially be displayed. Users wishing to have output Andres> that is rounded to match the previous behavior can set linkend="guc-extra-float-digits">extra_float_digits=0, Andres> which is no longer the default. Andres> Andres> Andres> Isn't it exactly the *other* way round? *Previously* we'd Andres> output additional trailing digits. The new algorithm instead Andres> will instead have *exactly* the required number of digits? Yeah, this wording is not right. But your description is also wrong. Previously we output values rounded to 6+d or 15+d digits where d=extra_float_digits, with d=0 being the default. Only clients that wanted exact results would set that to 3 instead. Now we output the minimum digits to get an exact result, which is usually 8 or 17 digits (sometimes less depending on the value, or 9 for the relatively rare float4 values that need it) for any extra_float_digits value > 0. Clients that set d=3 will therefore usually get one less digit than before, and the value they get will usually be slightly different (i.e. not the same value that they would have seen with d=2), but it should give them the same binary value after going through strtod() or strtof(). -- Andrew (irc:RhodiumToad)
Re: PG 12 draft release notes
Hi, Note that I've added a few questions to individuals involved with specific points. If you're in the To: list, please search for your name. On 2019-05-11 16:33:24 -0400, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > They are committed to git. Thanks! Migration to Version 12 There's a number of features in the compat section that are more general improvements with a side of incompatibility. Won't it be confusing to e.g. have have the ryu floating point conversion speedups in the compat section, but not in the "General Performance" section? Remove the special behavior of OID columns (Andres Freund, John Naylor) Should we mention that tables with OIDs have to have their oids removed before they can be upgraded? Refactor geometric functions and operators (Emre Hasegeli) This could lead to more accurate, but slightly different, results from previous releases. Restructure geometric types to handle NaN, underflow, overflow and division by zero more consistently (Emre Hasegeli) Improve behavior and error reporting for the line data type (Emre Hasegeli) Is that sufficient explanation? Feels like we need to expand a bit more. In particular, is it possible that a subset of the changes here require reindexing? Also, aren't three different entries a bit too much? Avoid performing unnecessary rounding of REAL and DOUBLE PRECISION values (Andrew Gierth) This dramatically speeds up processing of floating-point values but causes additional trailing digits to potentially be displayed. Users wishing to have output that is rounded to match the previous behavior can set extra_float_digits=0, which is no longer the default. Isn't it exactly the *other* way round? *Previously* we'd output additional trailing digits. The new algorithm instead will instead have *exactly* the required number of digits? Improve handling of partition dependency (Tom Lane) This prevents the creation of inconsistent partition hierarchies in rare cases. That seems not very informative for users? Improve speed of btree index insertions (Peter Geoghegan, Alexander Korotkov) The new code improves the space-efficiency of page splits, reduces locking overhead, and gives better performance for UPDATEs and DELETEs on indexes with many duplicates. Have new btree indexes sort duplicate index entries in heap-storage order (Peter Geoghegan, Heikki Linnakangas) Indexes pg_upgraded from previous releases will not have this ordering. I'm not sure that the grouping here is quite right. And the second entry probably should have some explanation about the benefits? Reduce locking requirements for index renaming (Peter Eisentraut) Should we specify the newly required lock level? Because it's quire relevant for users what exactly they're now able to do concurrently in operation? Allow common table expressions (CTE) to be inlined in later parts of the query (Andreas Karlsson, Andrew Gierth, David Fetter, Tom Lane) Specifically, CTEs are inlined if they are not recursive and are referenced only once later in the query. Inlining can be prevented by specifying MATERIALIZED, and forced by specifying NOT MATERIALIZED. Previously, CTEs were never inlined and were always evaluated before the rest of the query. Hm. Is it actually correct to say that "were always evaluated before the rest of the query."? My understanding is that that's not actually how they behaved. Materialization for CTE scans was on-demand (i.e. when needed by a CTE scan), and even for DML CTEs we'd only force the underlying query to completion at the end of the query? Add support for function selectivity (Tom Lane) Hm, that message doesn't seem like an accurate description of that commit (if anything it's a391ff3c?). Given that it all requires C hackery, perhaps we ought to move it to the source code section? And isn't the most important part of this set of changes commit 74dfe58a5927b22c744b29534e67bfdd203ac028 Author: Tom Lane Date: 2019-02-11 21:26:08 -0500 Allow extensions to generate lossy index conditions. Greatly reduce memory consumption of and function calls (Andres Freund, Tomas Vondra, Tom Lane) Grouping these three changes together makes no
Re: PG 12 draft release notes
On Tue, May 14, 2019 at 11:53:23AM +0200, nickb wrote: > On Sat, May 11, 2019, at 22:33, Bruce Momjian wrote: > > http://momjian.us/pgsql_docs/release-12.html > > There is a typo in E.1.3.1.1.: > > Expressions are evaluated at table partitioned table creation time. > First "table" seems to be excessive. Yep, fixed, thanks. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Sat, May 11, 2019, at 22:33, Bruce Momjian wrote: > http://momjian.us/pgsql_docs/release-12.html There is a typo in E.1.3.1.1.: > Expressions are evaluated at table partitioned table creation time. First "table" seems to be excessive. Regards, Nick.
Re: PG 12 draft release notes
On Mon, May 13, 2019 at 10:08:57AM +0300, Oleg Bartunov wrote: > On Mon, May 13, 2019 at 6:52 AM Bruce Momjian wrote: > > > > On Sun, May 12, 2019 at 09:49:40AM -0400, Bruce Momjian wrote: > > > On Sun, May 12, 2019 at 10:00:38AM +0300, Oleg Bartunov wrote: > > > > Bruce, > > > > > > > > I noticed that jsonpath in your version is mentioned only in functions > > > > chapter, but commit > > > > 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of > > > > SQL-2016 standard. We implemented JSON Path language as a jsonpath > > > > datatype with a bunch of support functions, our implementation > > > > supports 14 out of 15 features and it's the most complete > > > > implementation (we compared oracle, mysql and ms sql). > > > > > > Glad you asked. I was very confused about why a data type was added for > > > a new path syntax. Is it a new storage format for JSON, or something > > > else? I need help on this. > > > > I talked to Alexander Korotkov on chat about this. The data types are > > used as arguments to the functions, similar to how tsquery and tsvector > > are used for full text search. > > > > Therefore, the data types are not really useful on their own, but as > > support for path functions. However, path functions are more like JSON > > queries, rather than traditional functions, so it odd to list them under > > functions, but there isn't a more reasonable place to put them. > > > > Alexander researched how we listed full text search in the release notes > > that added the feature, but we had "General" category at that time that > > we don't have now. > > I attached slide about our Jsonpath implementation in Postgres, it > summarizes the reasons to have jsonpath data type. But my point was: > JSON Path is a part of SQL-2016 standard and I think it's worth to > mention it, not just a set of jsonb functions. So, are you suggesting we mention the jsonpath data type in the Data Type section, even though it is useless without jsonpath, which is in another section, or are you suggesting to move the jsonpath item to the Data Type section? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Mon, May 13, 2019 at 08:41:25AM +0200, Fabien COELHO wrote: > > Hello Bruce, > > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main docs, where > > appropriate. Our official developer docs will rebuild in a few hours. > > Pgbench entry "Improve pgbench error reporting with clearer messages and > return codes" is by Peter Eisentraut, not me. I just reviewed it. Thanks, fixed. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Mon, May 13, 2019 at 6:52 AM Bruce Momjian wrote: > > On Sun, May 12, 2019 at 09:49:40AM -0400, Bruce Momjian wrote: > > On Sun, May 12, 2019 at 10:00:38AM +0300, Oleg Bartunov wrote: > > > Bruce, > > > > > > I noticed that jsonpath in your version is mentioned only in functions > > > chapter, but commit > > > 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of > > > SQL-2016 standard. We implemented JSON Path language as a jsonpath > > > datatype with a bunch of support functions, our implementation > > > supports 14 out of 15 features and it's the most complete > > > implementation (we compared oracle, mysql and ms sql). > > > > Glad you asked. I was very confused about why a data type was added for > > a new path syntax. Is it a new storage format for JSON, or something > > else? I need help on this. > > I talked to Alexander Korotkov on chat about this. The data types are > used as arguments to the functions, similar to how tsquery and tsvector > are used for full text search. > > Therefore, the data types are not really useful on their own, but as > support for path functions. However, path functions are more like JSON > queries, rather than traditional functions, so it odd to list them under > functions, but there isn't a more reasonable place to put them. > > Alexander researched how we listed full text search in the release notes > that added the feature, but we had "General" category at that time that > we don't have now. I attached slide about our Jsonpath implementation in Postgres, it summarizes the reasons to have jsonpath data type. But my point was: JSON Path is a part of SQL-2016 standard and I think it's worth to mention it, not just a set of jsonb functions. > > -- > Bruce Momjian http://momjian.us > EnterpriseDB http://enterprisedb.com > > + As you are, so once was I. As I am, so you will be. + > + Ancient Roman grave inscription + -- Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Re: PG 12 draft release notes
Hello Bruce, I have posted a draft copy of the PG 12 release notes here: http://momjian.us/pgsql_docs/release-12.html They are committed to git. It includes links to the main docs, where appropriate. Our official developer docs will rebuild in a few hours. Pgbench entry "Improve pgbench error reporting with clearer messages and return codes" is by Peter Eisentraut, not me. I just reviewed it. -- Fabien.
Re: PG 12 draft release notes
On Mon, May 13, 2019 at 01:37:25PM +1200, David Rowley wrote: > On Sun, 12 May 2019 at 08:33, Bruce Momjian wrote: > > > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > I noticed a couple of different spellings of Álvaro's name. Loading > the file line by line into a table and crudely performing: > > select distinct name from (select > trim(regexp_split_to_table(substring(a, '\((.*?)\)'),',')) as name > from r where a like '%(%)%')a order by name; > > turned up variations in Michaël and Pavel's names That is a big help, thanks, applied. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Sun, May 12, 2019 at 09:49:40AM -0400, Bruce Momjian wrote: > On Sun, May 12, 2019 at 10:00:38AM +0300, Oleg Bartunov wrote: > > Bruce, > > > > I noticed that jsonpath in your version is mentioned only in functions > > chapter, but commit > > 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of > > SQL-2016 standard. We implemented JSON Path language as a jsonpath > > datatype with a bunch of support functions, our implementation > > supports 14 out of 15 features and it's the most complete > > implementation (we compared oracle, mysql and ms sql). > > Glad you asked. I was very confused about why a data type was added for > a new path syntax. Is it a new storage format for JSON, or something > else? I need help on this. I talked to Alexander Korotkov on chat about this. The data types are used as arguments to the functions, similar to how tsquery and tsvector are used for full text search. Therefore, the data types are not really useful on their own, but as support for path functions. However, path functions are more like JSON queries, rather than traditional functions, so it odd to list them under functions, but there isn't a more reasonable place to put them. Alexander researched how we listed full text search in the release notes that added the feature, but we had "General" category at that time that we don't have now. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Sun, May 12, 2019 at 10:49:07AM -0400, Jonathan Katz wrote: > Hi Bruce, > > On 5/11/19 4:33 PM, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main docs, where > > appropriate. Our official developer docs will rebuild in a few hours. > > Thank you for working on this, I know it's a gargantuan task. > > I have a small modification for a section entitled "Source Code" which > is a repeat of the previous section. Based on the bullet points in that > part, I thought "Documentation" might be a more appropriate name; please > see attached. Yes, I saw that myself and just updated it. Thanks. --- > > Thanks, > > Jonathan > diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml > index 5f5d1da33d..1bbd91a02e 100644 > --- a/doc/src/sgml/release-12.sgml > +++ b/doc/src/sgml/release-12.sgml > @@ -2617,7 +2617,7 @@ Require a C99-supported compiler, and > MSCV 2013 or later on < > > > > -Source Code > +Documentation > > > -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
On Sun, 12 May 2019 at 08:33, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html I noticed a couple of different spellings of Álvaro's name. Loading the file line by line into a table and crudely performing: select distinct name from (select trim(regexp_split_to_table(substring(a, '\((.*?)\)'),',')) as name from r where a like '%(%)%')a order by name; turned up variations in Michaël and Pavel's names The attached fixes. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services normalise_release_note_names.patch Description: Binary data
Re: PG 12 draft release notes
Hi Bruce, On 5/11/19 4:33 PM, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. Thank you for working on this, I know it's a gargantuan task. I have a small modification for a section entitled "Source Code" which is a repeat of the previous section. Based on the bullet points in that part, I thought "Documentation" might be a more appropriate name; please see attached. Thanks, Jonathan diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml index 5f5d1da33d..1bbd91a02e 100644 --- a/doc/src/sgml/release-12.sgml +++ b/doc/src/sgml/release-12.sgml @@ -2617,7 +2617,7 @@ Require a C99-supported compiler, and MSCV 2013 or later on < -Source Code +Documentation signature.asc Description: OpenPGP digital signature
Re: PG 12 draft release notes
On Sun, May 12, 2019 at 10:00:38AM +0300, Oleg Bartunov wrote: > Bruce, > > I noticed that jsonpath in your version is mentioned only in functions > chapter, but commit > 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of > SQL-2016 standard. We implemented JSON Path language as a jsonpath > datatype with a bunch of support functions, our implementation > supports 14 out of 15 features and it's the most complete > implementation (we compared oracle, mysql and ms sql). Glad you asked. I was very confused about why a data type was added for a new path syntax. Is it a new storage format for JSON, or something else? I need help on this. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 12 draft release notes
Bruce, I noticed that jsonpath in your version is mentioned only in functions chapter, but commit 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of SQL-2016 standard. We implemented JSON Path language as a jsonpath datatype with a bunch of support functions, our implementation supports 14 out of 15 features and it's the most complete implementation (we compared oracle, mysql and ms sql). On Sat, May 11, 2019 at 11:33 PM Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. > > -- > Bruce Momjian http://momjian.us > EnterpriseDB http://enterprisedb.com > > + As you are, so once was I. As I am, so you will be. + > + Ancient Roman grave inscription + > > -- Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
PG 12 draft release notes
I have posted a draft copy of the PG 12 release notes here: http://momjian.us/pgsql_docs/release-12.html They are committed to git. It includes links to the main docs, where appropriate. Our official developer docs will rebuild in a few hours. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
