Re: [HACKERS] Quorum commit for multiple synchronous replication.
On 08/22/2017 11:04 PM, Masahiko Sawada wrote: > WARNING: what you did is ok, but you might have wanted to do something else > > First of all, whether or not that can properly be called a warning is > highly debatable. Also, if you do that sort of thing to your spouse > and/or children, they call it "nagging". I don't think users will > like it any more than family members do. Realistically, we'll support the backwards-compatible syntax for 3-5 years. Which is fine. I suggest that we just gradually deprecate the old syntax from the docs, and then around Postgres 16 eliminate it. I posit that that's better than changing the meaning of the old syntax out from under people. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Quorum commit for multiple synchronous replication.
On 08/09/2017 10:49 PM, Michael Paquier wrote: > On Fri, Aug 4, 2017 at 8:19 AM, Masahiko Sawada wrote: >> On Fri, Jul 28, 2017 at 2:24 PM, Noah Misch wrote: >>> This item appears under "decisions to recheck mid-beta". If anyone is going >>> to push for a change here, now is the time. >> >> It has been 1 week since the previous mail. I though that there were >> others argued to change the behavior of old-style setting so that a >> quorum commit is chosen. If nobody is going to push for a change we >> can live with the current behavior? > > FWIW, I still see no harm in keeping backward-compatibility here, so I > am in favor of a statu-quo. > I am vaguely in favor of making quorum the default over "ordered". However, given that anybody using sync commit without understanding/customizing the setup is going to be sorry regardless, keeping backwards compatibility is acceptable. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Better error message for trying to drop a DB with open subscriptions?
All: The problem: postgres=# drop database bookdata; ERROR: database "bookdata" is being accessed by other users DETAIL: There is 1 other session using the database. postgres=# \c bookdata You are now connected to database "bookdata" as user "postgres". bookdata=# drop subscription wholedb; NOTICE: dropped replication slot "wholedb" on publisher DROP SUBSCRIPTION bookdata=# \c postgres You are now connected to database "postgres" as user "postgres". postgres=# drop database bookdata; DROP DATABASE Is there any easy way for us to detect that the "user" accessing the target database is actually a logical replication subscription, and give the DBA a better error message (e.g. "database 'bookdata' still has open subscrptions")? -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Notes on testing Postgres 10b1
On 06/09/2017 07:54 PM, Greg Stark wrote:
> On 7 June 2017 at 01:01, Josh Berkus wrote:
>> P3: apparently jsonb_to_tsvector with lang parameter isn't immutable?
>> This means that it can't be used for indexing:
>>
>> libdata=# create index bookdata_fts on bookdata using gin ((
>> to_tsvector('english',bookdata)));
>> ERROR: functions in index expression must be marked IMMUTABLE
>
> I don't have a machine handy to check on but isn't this a strange
> thing to do? Isn't there a GIN opclass on jsonb itself which would be
> the default if you didn't have that to_tsvector() call -- and which
> would also work properly with the jsonb operators?
>
The above is the documented way to create an FTS index on a JSONB field.
--
Josh Berkus
Containers & Databases Oh My!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] jsonb_to_tsvector should be immutable
Wanted to pull this out of my general report, because nobody seems to
have seen it:
P3: apparently jsonb_to_tsvector with lang parameter isn't immutable?
This means that it can't be used for indexing:
libdata=# create index bookdata_fts on bookdata using gin ((
to_tsvector('english',bookdata)));
ERROR: functions in index expression must be marked IMMUTABLE
... and indeed it's not:
select proname, prosrc, proargtypes, provolatile from pg_proc where
proname = 'to_tsvector';
proname | prosrc | proargtypes | provolatile
-++-+-
to_tsvector | jsonb_to_tsvector | 3802| s
to_tsvector | to_tsvector_byid | 3734 25 | i
to_tsvector | to_tsvector| 25 | s
to_tsvector | json_to_tsvector | 114 | s
to_tsvector | jsonb_to_tsvector_byid | 3734 3802 | s
to_tsvector | json_to_tsvector_byid | 3734 114| s
Both of the _byid functions should be marked immutable, no? Otherwise
how can users use the new functions for indexing?
--
Josh Berkus
Containers & Databases Oh My!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Notes on testing Postgres 10b1
On 06/07/2017 06:37 PM, Peter Eisentraut wrote: > On 6/7/17 21:19, Josh Berkus wrote: >> The user's first thought is going to be a network issue, or a bug, or >> some other problem, not a missing PK. Yeah, they can find that >> information in the logs, but only if they think to look for it in the >> first place, and in some environments (AWS, containers, etc.) logs can >> be very hard to access. > > You're not going to get very far with using this feature if you are not > looking in the logs for errors. These are asynchronously operating > background workers, so the only way they can communicate problems is > through the log. Well, we *could* provide a system view, as we now do for archiving, and for the same reasons. The issue isn't that the error detail is in the log. It's somehow letting the user know that they need to look at the log, as opposed to somewhere else. Consider that this is asynchonous for the user as well; they are likely to find out about the broken replication well after it happens, and thus have a lot of log to search through. Activity logs are a *terrible* UI for debugging systems problems. I realize that there is information it's hard for us to provide any other way. But the logs should be our "monitoring of last resort", where we put stuff after we've run out of ideas on where else to put it, because they are the hardest thing to access for a user. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Notes on testing Postgres 10b1
On 06/07/2017 07:01 PM, Petr Jelinek wrote: > On 08/06/17 03:50, Josh Berkus wrote: >> On 06/07/2017 06:25 PM, Petr Jelinek wrote: >>> On 08/06/17 03:19, Josh Berkus wrote: >>>> >>>> Peter and Petr: >>>> >>>> On 06/07/2017 05:24 PM, Peter Eisentraut wrote: >>>>> On 6/7/17 01:01, Josh Berkus wrote: >>>>>> * Having defaults on the various _workers all devolve from max_workers >>>>>> is also great. >>>>> >>>>> I'm not aware of anything like that happening. >>>>> >>>>>> P1. On the publishing node, logical replication relies on the *implied* >>>>>> correspondence of the application_name and the replication_slot both >>>>>> being named the same as the publication in order to associate a >>>>>> particular publication with a particular replication connection. >>>>>> However, there's absolutely nothing preventing me from also creating a >>>>>> binary replication connection by the same name It really seems like we >>>>>> need a field in pg_stat_replication or pg_replication_slots which lists >>>>>> the publication. >>>>> >>>>> I'm not quite sure what you are getting at here. The application_name >>>>> seen on the publisher side is the subscription name. You can create a >>>>> binary replication connection using the same application_name, but >>>>> that's already been possible before. But the publications don't care >>>>> about any of this. >>>> >>>> My point is that there is no system view where I can see, on the origin >>>> node, what subscribers are subscribing to which publications. You can >>>> kinda guess that from pg_stat_replication etc., but it's not dependable >>>> information. >>>> >>> >>> That's like wanting the foreign server to show you which foreign tables >>> exist on the local server. This is not a tightly coupled system and you >>> are able to setup both sides without them being connected to each other >>> at the time of setup, so there is no way publisher can know anything. >> >> Why wouldn't the publisher know who's connected once the replication >> connection as been made and the subscription has started? Or is it just >> a log position, and the publisher really has no idea how many >> publications are being consumed? >> > > Plugin knows while the connection exists, but that's the thing, it goes > through pluggable interface (that can be used by other plugins, without > publications) so there would have to be some abstracted way for plugins > to give some extra information for the pg_stat_replication or similar > view. I am afraid it's bit too late to design something like that in > PG10 cycle. OK, consider it a feature request for PG11, then. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Notes on testing Postgres 10b1
On 06/07/2017 06:25 PM, Petr Jelinek wrote: > On 08/06/17 03:19, Josh Berkus wrote: >> >> Peter and Petr: >> >> On 06/07/2017 05:24 PM, Peter Eisentraut wrote: >>> On 6/7/17 01:01, Josh Berkus wrote: >>>> * Having defaults on the various _workers all devolve from max_workers >>>> is also great. >>> >>> I'm not aware of anything like that happening. >>> >>>> P1. On the publishing node, logical replication relies on the *implied* >>>> correspondence of the application_name and the replication_slot both >>>> being named the same as the publication in order to associate a >>>> particular publication with a particular replication connection. >>>> However, there's absolutely nothing preventing me from also creating a >>>> binary replication connection by the same name It really seems like we >>>> need a field in pg_stat_replication or pg_replication_slots which lists >>>> the publication. >>> >>> I'm not quite sure what you are getting at here. The application_name >>> seen on the publisher side is the subscription name. You can create a >>> binary replication connection using the same application_name, but >>> that's already been possible before. But the publications don't care >>> about any of this. >> >> My point is that there is no system view where I can see, on the origin >> node, what subscribers are subscribing to which publications. You can >> kinda guess that from pg_stat_replication etc., but it's not dependable >> information. >> > > That's like wanting the foreign server to show you which foreign tables > exist on the local server. This is not a tightly coupled system and you > are able to setup both sides without them being connected to each other > at the time of setup, so there is no way publisher can know anything. Why wouldn't the publisher know who's connected once the replication connection as been made and the subscription has started? Or is it just a log position, and the publisher really has no idea how many publications are being consumed? -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Notes on testing Postgres 10b1
Peter and Petr: On 06/07/2017 05:24 PM, Peter Eisentraut wrote: > On 6/7/17 01:01, Josh Berkus wrote: >> * Having defaults on the various _workers all devolve from max_workers >> is also great. > > I'm not aware of anything like that happening. > >> P1. On the publishing node, logical replication relies on the *implied* >> correspondence of the application_name and the replication_slot both >> being named the same as the publication in order to associate a >> particular publication with a particular replication connection. >> However, there's absolutely nothing preventing me from also creating a >> binary replication connection by the same name It really seems like we >> need a field in pg_stat_replication or pg_replication_slots which lists >> the publication. > > I'm not quite sure what you are getting at here. The application_name > seen on the publisher side is the subscription name. You can create a > binary replication connection using the same application_name, but > that's already been possible before. But the publications don't care > about any of this. My point is that there is no system view where I can see, on the origin node, what subscribers are subscribing to which publications. You can kinda guess that from pg_stat_replication etc., but it's not dependable information. >> P2: If I create a subscription on a table with no primary key, I do not >> recieve a warning. There should be a warning, since in most cases such >> a subscription will not work. I suggest the text: >> >> "logical replication target relation "public.fines" has no primary key. >> Either create one, or set REPLICA IDENTITY index and set the published >> relation to REPLICA IDENTITY FULL." > > At that point, we don't know what is being published. If only inserts > are being published or REPLICA IDENTITY FULL is set, then it will work. > We don't want to give warnings about things that might not be true. > > More guidance on some of the potential failure cases would be good, but > it would need more refinement. Hmmm, yah, I see. Let me explain why this is a UX issue as-is though: 1. User forgets to create a PK on the subscriber node. 2. User starts a subscription to the tables. 3. Subscription is successful. 4. First update hits the publisher node. 5. Subscription fails and disconnects. The user's first thought is going to be a network issue, or a bug, or some other problem, not a missing PK. Yeah, they can find that information in the logs, but only if they think to look for it in the first place, and in some environments (AWS, containers, etc.) logs can be very hard to access. We really need the subscription to fail at step (2), not wait for the first update to fail. And if it doesn't fail at step 2, then we should at least give a warning. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Notes on testing Postgres 10b1
Folks,
I've put together some demos on PostgreSQL 10beta1. Here's a few
feedback notes based on my experience with it.
Things I tested
* Logical replication pub/sub with replicating only two tables out of a
12-table FK heirarchy, including custom data types
* Partitioning a log-structured table, including a range type, exclusion
constraint, and foreign key.
* Various Parallel index queries on a 100m-row pgbench table
* Full text JSON search in a books database
* SCRAM authentication for local connections and replication
Positive changes beyond the obvious
---
* Yay defaults with replication on!
* Having defaults on the various _workers all devolve from max_workers
is also great.
* Constraint exclusion + partitioning Just Worked.
Questions
--
Q1. Why does wal_level default to "replica" and not "logical"?
Q2: I thought we were going to finally change the pg_dump default to
"custom" format in this release? No?
Problems
P1. On the publishing node, logical replication relies on the *implied*
correspondence of the application_name and the replication_slot both
being named the same as the publication in order to associate a
particular publication with a particular replication connection.
However, there's absolutely nothing preventing me from also creating a
binary replication connection by the same name It really seems like we
need a field in pg_stat_replication or pg_replication_slots which lists
the publication.
P2: If I create a subscription on a table with no primary key, I do not
recieve a warning. There should be a warning, since in most cases such
a subscription will not work. I suggest the text:
"logical replication target relation "public.fines" has no primary key.
Either create one, or set REPLICA IDENTITY index and set the published
relation to REPLICA IDENTITY FULL."
P3: apparently jsonb_to_tsvector with lang parameter isn't immutable?
This means that it can't be used for indexing:
libdata=# create index bookdata_fts on bookdata using gin ((
to_tsvector('english',bookdata)));
ERROR: functions in index expression must be marked IMMUTABLE
... and indeed it's not:
select proname, prosrc, proargtypes, provolatile from pg_proc where
proname = 'to_tsvector';
proname | prosrc | proargtypes | provolatile
-++-+-
to_tsvector | jsonb_to_tsvector | 3802| s
to_tsvector | to_tsvector_byid | 3734 25 | i
to_tsvector | to_tsvector| 25 | s
to_tsvector | json_to_tsvector | 114 | s
to_tsvector | jsonb_to_tsvector_byid | 3734 3802 | s
to_tsvector | json_to_tsvector_byid | 3734 114| s
... can we fix that?
--
Josh Berkus
Containers & Databases Oh My!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Missing feature in Phrase Search?
Oleg, Teodor, folks: I was demo'ing phrase search for a meetup yesterday, and the user feedback I got showed that there's a missing feature with phrase search. Let me explain by example: 'fix <-> error' will match 'fixed error', 'fixing error' but not 'fixed language error' or 'fixed a small error' 'fix <2> error' will match 'fixed language error', but not 'fixing error' or 'fixed a small error' 'fix <3> error' will match 'fixed a small error', but not any of the other strings. This is because the # in <#> is an exact match. Seems like we could really use a way for users to indicate that they want a range of word gaps. Like, in the example above, users could search on: 'fix <1:3> error' ... which would search for any phrase where "error" followed "fix" by between 1 and 3 words. Not wedded to any particular syntax for that, of course. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 2017-03 CF Closed
On 04/08/2017 09:12 AM, Tom Lane wrote: > Michael Paquier writes: >> On Sat, Apr 8, 2017 at 11:27 PM, Andres Freund wrote: >>> Thanks for your work on managing the fest! > >> +1. Great work! > > Seconded. That's a huge amount of generally-underappreciated work. > And only a week over schedule. Good work. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL/JSON in PostgreSQL
On 03/09/2017 10:12 AM, Sven R. Kunze wrote: > On 08.03.2017 20:52, Magnus Hagander wrote: >> On Wed, Mar 8, 2017 at 11:48 AM, Peter van Hardenberg > <mailto:p...@pvh.ca>> wrote: >> >> Small point of order: YAML is not strictly a super-set of JSON. >> >> Editorializing slightly, I have not seen much interest in the >> world for YAML support though I'd be interested in evidence to the >> contrary. >> >> >> The world of configuration management seems to for some reason run off >> YAML, but that's the only places I've seen it recently (ansible, >> puppet etc). > > SaltStack uses YAML for their tools, too. I personally can empathize > with them (as a user of configuration management) about this as writing > JSON would be nightmare with all the quoting, commas, curly braces etc. > But that's my own preference maybe. > > (Btw. does "run off" mean like or avoid? At least my dictionaries tend > to the latter.) Yes, but automated tools can easily convert between JSON and newline-delimited YAML and back. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for changes to recovery.conf API
On 03/02/2017 07:13 AM, David Steele wrote: > Hi Simon, > > On 2/25/17 2:43 PM, Simon Riggs wrote: >> On 25 February 2017 at 13:58, Michael Paquier >> wrote: >> >>> - trigger_file is removed. >>> FWIW, my only complain is about the removal of trigger_file, this is >>> useful to detect a trigger file on a different partition that PGDATA! >>> Keeping it costs also nothing.. >> >> Sorry, that is just an error of implementation, not intention. I had >> it on my list to keep, at your request. >> >> New version coming up. > > Do you have an idea when the new version will be available? Please? Having yet another PostgreSQL release go by without fixing recovery.conf would make me very sad. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Possible spelling fixes
0001-spelling-comments.patch Description: Binary data 0002-spelling-strings.patch Description: Binary data 0003-spelling-sgml.patch Description: Binary data 0004-spelling-variables.patch Description: Binary data 0005-spelling-misc.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Possible spelling fixes
Peter Eisentraut wrote: > Yes, some of that was committed, and some comments were offered. If > there is more to do, please send a rebased patch set. Conflicting comments were offered. And Heikki requested I send along the remainder. Which I did. Only one of those patches would have been impacted by the conflicting comments. The patches in this thread still applied today: spelling: comments -- conflicting comments about NUL/NULL spelling: strings -- no comments / applied cleanly spelling: variables -- no comments / applied cleanly spelling: misc -- no comments / applied cleanly spelling: api -- no comments until today / applied cleanly, may end up being dropped I want to thank Heikki for the initial acceptance and Alvaro and David for their additional comments. I'm not going to send a new submission before tonight. If anyone else wants to make comments before I resubmit, I welcome them... For reference, my current work queue is here: https://github.com/postgres/postgres/compare/master...jsoref:spelling-words?expand=1 The rebased version of the patches that were submitted but ignored are here: https://github.com/postgres/postgres/compare/master...jsoref:spelling?expand=1 I haven't updated them to reflect my work queue, as selecting things into those bundles is a not particularly pleasant, and thus a step I want to do as few times as possible. One thing that would be helpful is if someone could comment on: https://github.com/jsoref/postgres/commit/9050882d601134ea1ba26f77ce5f1aaed75418de -#undef SH_ITERTOR +#undef SH_ITERATOR It's unclear to me what that line is/was doing. It's possible that it could be removed entirely instead of having its spelling changed. If the line is trying to guard against a previous version of the code, which is no longer active, then it deserves a comment. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Possible spelling fixes
I'll include it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Possible spelling fixes
I understood that they were git commits. I could have excluded the file but opted not to in case people were willing to take a small drift -- the SHAs are what someone needs to review the commit, and personally, I'd rather read something without typos than with -- especially in a summary. But, I'll tentatively exclude the lines containing SHAs. do you want: @@ -3116,7 +3116,7 @@ 2016-02-17 [a5c43b886] Add new system vi This view exposes the same information available from -the pg_config comand-line utility, +the pg_config command-line utility, namely assorted compile-time configuration information for PostgreSQL. or should I exclude the file entirely? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Possible spelling fixes
I can easily drop the shutdown* items; The reason for rowMarks is consistency with lr_arowMarks. I'll tentatively exclude that from any resubmission I make tonight... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Possible spelling fixes
On Mar 1, 2017 9:06 AM, "Peter Eisentraut" wrote: On 2/6/17 06:03, Heikki Linnakangas wrote: > Ah, yes please. Post them over and I'll have a look at those as well. This thread is in the commit fest, but I think there is no current patch. I sent email on the 6th with a number of patches as attachments...
Re: [HACKERS] Proposal for changes to recovery.conf API
On 02/26/2017 12:55 AM, Robert Haas wrote: > On Wed, Jan 11, 2017 at 11:23 PM, Simon Riggs wrote: >>> I think the issue was that some people didn't want configuration files >>> in the data directory. By removing recovery.conf we accomplish that. >>> Signal/trigger files are not configuration (or at least it's much easier >>> to argue that), so I think having them in the data directory is fine. >> >> There were a considerable number of people that pushed to make the >> data directory non-user writable, which is where the signal directory >> came from. > > Specifically, it's a problem for Debian's packaging conventions, > right? The data directory can contain anything that the server itself > will write, but configuration files that are written for the server to > read are supposed to go in some external location dictated by Debian's > packaging policy. > > Things like trigger files aren't configuration files per se, so maybe > it's OK if those still get written into the data directory. Even if > not, that seems like a separate patch. In my view, based on Michael's > description of what the current patch version does, it's a clear step > forward. Other steps can be taken at another time, if required. > >From the perspective of containerized Postgres, you want config files to go into one (non-writeable) directory, and anything which is writeable by the DB server to go into another directory (and preferably, a single directory). A trigger file (that is, assuming an empty one, and recovery config merged with pg.conf) would thus be writeable, non-configuration data which goes in the data directory. Users manually writing the trigger file doesn't show up as a problem since, in a containerized environment, they can't. It's either written by postgres itself, or by management software which runs as the postgres user. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Official adoption of PGXN
On 02/14/2017 12:05 PM, Tom Lane wrote: > Jim Nasby writes: >> First, just to clarify: my reasons for proposing "core adoption" of PGXN >> are not technical in nature. > > What do you think "core adoption" means? Surely not that anything > associated with PGXN would be in the core distro. One part of this would need to be having a designated committee of the Postgres community pick a set of "blessed" extensions for packagers to package. Right now, contrib serves that purpose (badly). One of the reasons we haven't dealt with the extension distribution problem is that nobody wanted to take on the issue of picking a list of blessed extensions. > >> Right now contrib is serving two completely separate purposes: > >> 1) location for code that (for technical reasons) should be tied to >> specific PG versions >> 2) indication of "official endorsement" of a module by the community > > This argument ignores what I think is the real technical reason for > keeping contrib, which is to have a set of close-at-hand test cases > for extension and hook mechanisms. Certainly, not every one of the > existing contrib modules is especially useful for that purpose, but > quite a few of them are. Yes. But there's a bunch that aren't, and those are the ones which we previously discussed, the ones with indifferent maintenance like ISN and Intarray. You have to admit that it seems really strange in the eyes of a new user that ISN is packaged with PostgreSQL, whereas better-written and more popular extensions (like plv8, pg_partman or pgq) are not. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Removal of deprecated views pg_user, pg_group, pg_shadow
On 02/13/2017 11:00 AM, Robert Haas wrote: > My big objection to removing these views is that it will break pgAdmin > 3, which uses all three of these views. I understand that the pgAdmin > community is now moving away from pgAdmin 3 and toward pgAdmin 4, but > I bet that pgAdmin 3 still has significant usage and will continue to > have significant usage for at least a year or three. When it's > thoroughly dead, then I think we can go ahead and do this, unless > there are other really important tools still depending on those views, > but it's only been 3 months since the final pgAdmin 3 release. How long would you suggest is appropriate? Postgres 11? 12? Let's set a target date; that way we can communicate it more than once. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] removing tsearch2
On 02/10/2017 10:18 AM, Peter Geoghegan wrote: > On Fri, Feb 10, 2017 at 3:28 AM, Robert Haas wrote: >>>> Works for me. >>> >>> +1 >> >> OK, that's three votes in favor of removing tsearch2 (from core, >> anyone who wants it can maintain a copy elsewhere). > > +1. > > I'd also be in favor of either removing contrib/isn, or changing it so > that the ISBN country code prefix enforcement went away. That would > actually not imply and real loss of functionality from a practical > perspective, since you can still enforce that the check digit is > correct without any of that. I think that the existing design of some > parts of contrib/isn is just horrible. +1 to quick-fix it, -1 to just delete it. There's a bunch of these things in /contrib which really ought to be PGXN extensions (also CUBE, earthdistance, etc.). However, one of the steps in that would be getting the mainstream platforms to package them so that users have a reasonable upgrade path, so I would not propose doing it for 10. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] removing tsearch2
On 02/10/2017 06:41 AM, David Steele wrote: > On 2/10/17 6:28 AM, Robert Haas wrote: >> On Thu, Feb 9, 2017 at 7:37 PM, Andres Freund wrote: >>> On 2017-02-09 19:19:21 -0500, Stephen Frost wrote: >>>> * Robert Haas (robertmh...@gmail.com) wrote: >>>>> On Thu, Feb 9, 2017 at 4:24 PM, Tom Lane wrote: >>>>>> Also, our experience with contrib/tsearch2 suggests that the extension >>>>>> shouldn't be part of contrib, because we have zero track record of >>>>>> getting >>>>>> rid of stuff in contrib, no matter how dead it is. >>>>> >>>>> Let's nuke tsearch2 to remove this adverse precedent, and then add the >>>>> new thing. >>>>> >>>>> Anybody who still wants tsearch2 can go get it from an old version, or >>>>> somebody can maintain a fork on github. >>>> >>>> Works for me. >>> >>> +1 >> >> OK, that's three votes in favor of removing tsearch2 (from core, >> anyone who wants it can maintain a copy elsewhere). Starting a new >> thread to make sure we collect all the relevant votes, but I really, >> really think it's past time for this to go away. The last actual >> change to tsearch2 which wasn't part of a wider cleanup was >> 3ca7eddbb7c4803729d385a0c9535d8a972ee03f in January 2009, so it's been >> 7 years since there's been any real work done on this -- and the >> release where we brought tsearch into core is now EOL, plus three more >> releases besides. > > +1 > +1 -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal
On 02/09/2017 05:19 PM, Michael Paquier wrote: > On Fri, Feb 10, 2017 at 10:16 AM, Stephen Frost wrote: >>> As someone mentioned, forcing a user to install an extension makes >>> the deprecation visible. Another option would be to have the backend >>> spit out a WARNING the first time you access anything that's >>> deprecated. Both of those are pertinent reminders to people that >>> they need to change their tools. >> >> Ugh. Please, no. Hacking up the backend to recognize that a given >> query is referring to a deprecated view and then throwing a warning on >> it is just plain ugly. >> >> Let's go one step further, and throw an ERROR if someone tries to query >> these views instead. > > FWIW, I am of the opinion to just nuke them as the "soft of" > deprecation period has been very long. Applications should have > switched to pg_authid and pg_roles long ago already. > We will definitely break a lot of client code by removing these -- I know that, deprecated or not, a lot of infrequently-updated driver/orm/GUI code still refers to pg_shadow/pg_user. I think Postgres 10 is the right time to break that code (I mean, we have to do it someday, and we're already telling people about breakage in 10), but be aware that there will be shouting and carrying on. -1 on a warning. Very little code today which references the deprecated code is interactive, so who's going to see the warning? -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal
On 02/09/2017 12:53 PM, Stephen Frost wrote: > * Josh Berkus (j...@berkus.org) wrote: >> On 02/09/2017 12:42 PM, Stephen Frost wrote: >>> * Josh Berkus (j...@berkus.org) wrote: >>>> On 02/09/2017 11:08 AM, Tom Lane wrote: >>>>> Agreed, let's just get it done. >>>>> >>>>> Although this doesn't really settle whether we ought to do 3a (with >>>>> backwards-compatibility function aliases in core) or 3b (without 'em). >>>>> Do people want to re-vote, understanding that those are the remaining >>>>> choices? >>>> >>>> Does 3a) mean keeping the aliases more-or-less forever? >>>> >>>> If not, I vote for 3b. If we're going to need to break stuff, let's >>>> just do it. >>>> >>>> If we can keep the aliases for 6-10 years, then I see no reason not to >>>> have them (3a). They're not exactly likely to conflict with user-chosen >>>> names. >>> >>> When we remove pg_shadow, then I'll be willing to agree that maybe we >>> can start having things in PG for a couple releases that are just for >>> backwards-compatibility and will actually be removed later. >>> >>> History has shown that's next to impossible, however. >> >> That's why I said 6-10 years. If we're doing 3a, realistically we're >> supporting it until PostgreSQL 16, at least, and more likely 20. I'm OK >> with that. > > Uh, to be clear, I think it's an entirely bad thing that we've had those > views and various other cruft hang around for over 10 years. > > And removing them today will probably still have people crying about how > pgAdmin3 and other things still use them. > >> What I'm voting against is the idea that we'll have aliases in core, but >> remove them in two releases. Either that's unrealistic, or it's just >> prolonging the pain. > > Waiting 10+ years doesn't make the pain go away when it comes to > removing things like that. Sure it does. That's two whole generations of client tools. For example, at that point, pgAdmin3 won't reliably run on any supported platform, so it won't be a problem if we break it. If we clearly mark the old function names as deprecated aliases, client tools will gradually move to the new names. Counter-argument: moving the directory is going to break many tools anyway, so why bother with function aliases? -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal
On 02/09/2017 12:42 PM, Stephen Frost wrote: > * Josh Berkus (j...@berkus.org) wrote: >> On 02/09/2017 11:08 AM, Tom Lane wrote: >>> Agreed, let's just get it done. >>> >>> Although this doesn't really settle whether we ought to do 3a (with >>> backwards-compatibility function aliases in core) or 3b (without 'em). >>> Do people want to re-vote, understanding that those are the remaining >>> choices? >> >> Does 3a) mean keeping the aliases more-or-less forever? >> >> If not, I vote for 3b. If we're going to need to break stuff, let's >> just do it. >> >> If we can keep the aliases for 6-10 years, then I see no reason not to >> have them (3a). They're not exactly likely to conflict with user-chosen >> names. > > When we remove pg_shadow, then I'll be willing to agree that maybe we > can start having things in PG for a couple releases that are just for > backwards-compatibility and will actually be removed later. > > History has shown that's next to impossible, however. That's why I said 6-10 years. If we're doing 3a, realistically we're supporting it until PostgreSQL 16, at least, and more likely 20. I'm OK with that. What I'm voting against is the idea that we'll have aliases in core, but remove them in two releases. Either that's unrealistic, or it's just prolonging the pain. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal
On 02/09/2017 11:08 AM, Tom Lane wrote: > Agreed, let's just get it done. > > Although this doesn't really settle whether we ought to do 3a (with > backwards-compatibility function aliases in core) or 3b (without 'em). > Do people want to re-vote, understanding that those are the remaining > choices? Does 3a) mean keeping the aliases more-or-less forever? If not, I vote for 3b. If we're going to need to break stuff, let's just do it. If we can keep the aliases for 6-10 years, then I see no reason not to have them (3a). They're not exactly likely to conflict with user-chosen names. -- Josh Berkus Containers & Databases Oh My! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Possible spelling fixes
diff --git a/src/backend/access/transam/xlog.c
b/src/backend/access/transam/xlog.c
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6083,7 +6083,7 @@ StartupXLOG(void)
ereport(LOG,
(errmsg("database system was shut down in
recovery at %s",
str_time(ControlFile->time;
- else if (ControlFile->state == DB_SHUTDOWNING)
+ else if (ControlFile->state == DB_SHUTTINGDOWN)
ereport(LOG,
(errmsg("database system shutdown was
interrupted; last known up at %s",
str_time(ControlFile->time;
@@ -8398,7 +8398,7 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ ControlFile->state = DB_SHUTTINGDOWN;
ControlFile->time = (pg_time_t) time(NULL);
UpdateControlFile();
LWLockRelease(ControlFileLock);
diff --git a/src/backend/executor/nodeLockRows.c
b/src/backend/executor/nodeLockRows.c
--- a/src/backend/executor/nodeLockRows.c
+++ b/src/backend/executor/nodeLockRows.c
@@ -349,7 +349,7 @@ ExecInitLockRows(LockRows *node, EState
{
LockRowsState *lrstate;
Plan *outerPlan = outerPlan(node);
- List *epq_arowmarks;
+ List *epq_arowMarks;
ListCell *lc;
/* check for unsupported flags */
@@ -398,7 +398,7 @@ ExecInitLockRows(LockRows *node, EState
* built the global list of ExecRowMarks.)
*/
lrstate->lr_arowMarks = NIL;
- epq_arowmarks = NIL;
+ epq_arowMarks = NIL;
foreach(lc, node->rowMarks)
{
PlanRowMark *rc = castNode(PlanRowMark, lfirst(lc));
@@ -425,12 +425,12 @@ ExecInitLockRows(LockRows *node, EState
if (RowMarkRequiresRowShareLock(erm->markType))
lrstate->lr_arowMarks = lappend(lrstate->lr_arowMarks,
aerm);
else
- epq_arowmarks = lappend(epq_arowmarks, aerm);
+ epq_arowMarks = lappend(epq_arowMarks, aerm);
}
/* Now we have the info needed to set up EPQ state */
EvalPlanQualInit(&lrstate->lr_epqstate, estate,
-outerPlan, epq_arowmarks,
node->epqParam);
+outerPlan, epq_arowMarks,
node->epqParam);
return lrstate;
}
diff --git a/src/backend/executor/nodeModifyTable.c
b/src/backend/executor/nodeModifyTable.c
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1471,7 +1471,7 @@ ExecModifyTable(ModifyTableState *node)
junkfilter = resultRelInfo->ri_junkFilter;
estate->es_result_relation_info = resultRelInfo;
EvalPlanQualSetPlan(&node->mt_epqstate,
subplanstate->plan,
-
node->mt_arowmarks[node->mt_whichplan]);
+
node->mt_arowMarks[node->mt_whichplan]);
continue;
}
else
@@ -1653,7 +1653,7 @@ ExecInitModifyTable(ModifyTable *node, E
mtstate->mt_plans = (PlanState **) palloc0(sizeof(PlanState *) *
nplans);
mtstate->resultRelInfo = estate->es_result_relations +
node->resultRelIndex;
- mtstate->mt_arowmarks = (List **) palloc0(sizeof(List *) * nplans);
+ mtstate->mt_arowMarks = (List **) palloc0(sizeof(List *) * nplans);
mtstate->mt_nplans = nplans;
mtstate->mt_onconflict = node->onConflictAction;
mtstate->mt_arbiterindexes = node->arbiterIndexes;
@@ -1975,7 +1975,7 @@ ExecInitModifyTable(ModifyTable *node, E
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
- mtstate->mt_arowmarks[i] =
lappend(mtstate->mt_arowmarks[i], aerm);
+ mtstate->mt_arowMarks[i] =
lappend(mtstate->mt_arowMarks[i], aerm);
}
}
@@ -1983,7 +1983,7 @@ ExecInitModifyTable(ModifyTable *node, E
mtstate->mt_whichplan = 0;
subplan = (Plan *) linitial(node->plans);
EvalPlanQualSetPlan(&mtstate->mt_epqstate, subplan,
- mtstate->mt_arowmarks[0]);
+ mtstate->mt_arowMarks[0]);
/*
* Initialize the junk filter(s) if needed. INSERT queries need a
filter
diff --git a/src/bin/pg_controldata/pg_controldata.c
b/src/bin/pg_controldata/pg_controldata.c
--- a/src/bin/pg_controldata/pg_controldata.c
++
Re: [HACKERS] Possible spelling fixes
diff --git a/src/test/regress/sql/plpgsql.sql b/src/test/regress/sql/plpgsql.sql --- a/src/test/regress/sql/plpgsql.sql +++ b/src/test/regress/sql/plpgsql.sql @@ -2219,15 +2219,15 @@ drop type eitype cascade; -- SQLSTATE and SQLERRM test -- -create function excpt_test1() returns void as $$ +create function except_test1() returns void as $$ begin raise notice '% %', sqlstate, sqlerrm; end; $$ language plpgsql; -- should fail: SQLSTATE and SQLERRM are only in defined EXCEPTION -- blocks -select excpt_test1(); - -create function excpt_test2() returns void as $$ +select except_test1(); + +create function except_test2() returns void as $$ begin begin begin @@ -2236,9 +2236,9 @@ begin end; end; $$ language plpgsql; -- should fail -select excpt_test2(); - -create function excpt_test3() returns void as $$ +select except_test2(); + +create function except_test3() returns void as $$ begin begin raise exception 'user exception'; @@ -2257,19 +2257,19 @@ begin raise notice '% %', sqlstate, sqlerrm; end; end; $$ language plpgsql; -select excpt_test3(); - -create function excpt_test4() returns text as $$ +select except_test3(); + +create function except_test4() returns text as $$ begin begin perform 1/0; exception when others then return sqlerrm; end; end; $$ language plpgsql; -select excpt_test4(); - -drop function excpt_test1(); -drop function excpt_test2(); -drop function excpt_test3(); -drop function excpt_test4(); +select except_test4(); + +drop function except_test1(); +drop function except_test2(); +drop function except_test3(); +drop function except_test4(); -- parameters of raise stmt can be expressions create function raise_exprs() returns void as $$ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Possible spelling fixes
diff --git a/src/backend/access/transam/multixact.c
b/src/backend/access/transam/multixact.c
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3342,7 +3342,7 @@ pg_get_multixact_members(PG_FUNCTION_ARG
} mxact;
MultiXactId mxid = PG_GETARG_UINT32(0);
mxact *multi;
- FuncCallContext *funccxt;
+ FuncCallContext *funcctx;
if (mxid < FirstMultiXactId)
ereport(ERROR,
@@ -3354,8 +3354,8 @@ pg_get_multixact_members(PG_FUNCTION_ARG
MemoryContext oldcxt;
TupleDesc tupdesc;
- funccxt = SRF_FIRSTCALL_INIT();
- oldcxt = MemoryContextSwitchTo(funccxt->multi_call_memory_ctx);
+ funcctx = SRF_FIRSTCALL_INIT();
+ oldcxt = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
multi = palloc(sizeof(mxact));
/* no need to allow for old values here */
@@ -3369,14 +3369,14 @@ pg_get_multixact_members(PG_FUNCTION_ARG
TupleDescInitEntry(tupdesc, (AttrNumber) 2, "mode",
TEXTOID, -1, 0);
- funccxt->attinmeta = TupleDescGetAttInMetadata(tupdesc);
- funccxt->user_fctx = multi;
+ funcctx->attinmeta = TupleDescGetAttInMetadata(tupdesc);
+ funcctx->user_fctx = multi;
MemoryContextSwitchTo(oldcxt);
}
- funccxt = SRF_PERCALL_SETUP();
- multi = (mxact *) funccxt->user_fctx;
+ funcctx = SRF_PERCALL_SETUP();
+ multi = (mxact *) funcctx->user_fctx;
while (multi->iter < multi->nmembers)
{
@@ -3386,16 +3386,16 @@ pg_get_multixact_members(PG_FUNCTION_ARG
values[0] = psprintf("%u", multi->members[multi->iter].xid);
values[1] =
mxstatus_to_string(multi->members[multi->iter].status);
- tuple = BuildTupleFromCStrings(funccxt->attinmeta, values);
+ tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
}
if (multi->nmembers > 0)
pfree(multi->members);
pfree(multi);
- SRF_RETURN_DONE(funccxt);
+ SRF_RETURN_DONE(funcctx);
}
diff --git a/src/backend/access/transam/xlog.c
b/src/backend/access/transam/xlog.c
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -851,7 +851,7 @@ static bool InstallXLogFileSegment(XLogS
bool find_free, XLogSegNo max_segno,
bool use_lock);
static int XLogFileRead(XLogSegNo segno, int emode, TimeLineID tli,
-int source, bool notexistOk);
+int source, bool notfoundOk);
static int XLogFileReadAnyTLI(XLogSegNo segno, int emode, int source);
static int XLogPageRead(XLogReaderState *xlogreader, XLogRecPtr targetPagePtr,
int reqLen, XLogRecPtr targetRecPtr, char *readBuf,
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -486,7 +486,7 @@ static void agg_fill_hash_table(AggState
static TupleTableSlot *agg_retrieve_hash_table(AggState *aggstate);
static Datum GetAggInitVal(Datum textInitVal, Oid transtype);
static void build_pertrans_for_aggref(AggStatePerTrans pertrans,
- AggState *aggsate, EState
*estate,
+ AggState *aggstate, EState
*estate,
Aggref *aggref, Oid
aggtransfn, Oid aggtranstype,
Oid aggserialfn, Oid
aggdeserialfn,
Datum initValue, bool
initValueIsNull,
diff --git a/src/backend/replication/walreceiverfuncs.c
b/src/backend/replication/walreceiverfuncs.c
--- a/src/backend/replication/walreceiverfuncs.c
+++ b/src/backend/replication/walreceiverfuncs.c
@@ -322,7 +322,7 @@ GetReplicationApplyDelay(void)
longsecs;
int usecs;
- TimestampTz chunckReplayStartTime;
+ TimestampTz chunkReplayStartTime;
SpinLockAcquire(&walrcv->mutex);
receivePtr = walrcv->receivedUpto;
@@ -333,12 +333,12 @@ GetReplicationApplyDelay(void)
if (receivePtr == replayPtr)
return 0;
- chunckReplayStartTime = GetCurrentChunkReplayStartTime();
+ chunkReplayStartTime = GetCurrentChunkReplayStartTime();
- if (chunckReplayStartTime == 0)
+ if (chunkReplayStartTime == 0)
return -1;
- TimestampDifference(chunckR
Re: [HACKERS] Possible spelling fixes
diff --git a/contrib/ltree/ltxtquery_io.c b/contrib/ltree/ltxtquery_io.c
--- a/contrib/ltree/ltxtquery_io.c
+++ b/contrib/ltree/ltxtquery_io.c
@@ -96,7 +96,7 @@ gettoken_query(QPRS_STATE *state, int32
if (*flag)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
-
errmsg("modificators syntax error")));
+
errmsg("modifiers syntax error")));
*lenval += charlen;
}
else if (charlen == 1 && t_iseq(state->buf,
'%'))
diff --git a/doc/src/sgml/biblio.sgml b/doc/src/sgml/biblio.sgml
--- a/doc/src/sgml/biblio.sgml
+++ b/doc/src/sgml/biblio.sgml
@@ -295,7 +295,7 @@ ssimk...@ag.or.at
April, 1990
University of California
- Berkely, California
+ Berkeley, California
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1225,7 +1225,7 @@ postgres 27093 0.0 0.0 30096 2752
MessageQueuePutMessage
- Waiting to write a protoocol message to a shared message
queue.
+ Waiting to write a protocol message to a shared message
queue.
MessageQueueReceive
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -5882,7 +5882,7 @@ TupleData
-Idenfifies the data as NULL value.
+Identifies the data as NULL value.
@@ -5895,7 +5895,7 @@ TupleData
-Idenfifies unchanged TOASTed value (the actual value is not
+Identifies unchanged TOASTed value (the actual value is not
sent).
@@ -5909,7 +5909,7 @@ TupleData
-Idenfifies the data as text formatted value.
+Identifies the data as text formatted value.
diff --git a/doc/src/sgml/ref/create_subscription.sgml
b/doc/src/sgml/ref/create_subscription.sgml
--- a/doc/src/sgml/ref/create_subscription.sgml
+++ b/doc/src/sgml/ref/create_subscription.sgml
@@ -131,7 +131,7 @@ CREATE SUBSCRIPTION option -g
-to specify role membership (Chistopher Browne)
+to specify role membership (Christopher Browne)
diff --git a/doc/src/sgml/release-9.6.sgml b/doc/src/sgml/release-9.6.sgml
--- a/doc/src/sgml/release-9.6.sgml
+++ b/doc/src/sgml/release-9.6.sgml
@@ -2783,7 +2783,7 @@ 2015-12-30 [e84290823] Avoid useless tru
Improve full-text search to support
@@ -4612,7 +4612,7 @@ 2016-02-16 [fc1ae7d2e] Change ecpg lexer
Add a --strict-names option
@@ -5609,7 +5609,7 @@ 2016-03-16 [f576b17cd] Add word_similari
diff --git a/doc/src/sgml/release-old.sgml b/doc/src/sgml/release-old.sgml
--- a/doc/src/sgml/release-old.sgml
+++ b/doc/src/sgml/release-old.sgml
@@ -5737,8 +5737,8 @@ fix rtree for use in inner scan (Vadim)
fix gist for use in inner scan, cleanups (Vadim, Andrea)
avoid unnecessary local buffers allocation (Vadim, Massimo)
fix local buffers leak in transaction aborts (Vadim)
-fix file manager memmory leaks, cleanups (Vadim, Massimo)
-fix storage manager memmory leaks (Vadim)
+fix file manager memory leaks, cleanups (Vadim, Massimo)
+fix storage manager memory leaks (Vadim)
fix btree duplicates handling (Vadim)
fix deleted rows reincarnation caused by vacuum (Vadim)
fix SELECT varchar()/char() INTO TABLE made zero-length fields(Bruce)
@@ -5904,7 +5904,7 @@ European date format now set when postma
Execute lowercase function names if not found with exact case
Fixes for aggregate/GROUP processing, allow 'select sum(func(x),sum(x+y) from
z'
Gist now included in the distribution(Marc)
-Idend authentication of local users(Bryan)
+Ident authentication of local users(Bryan)
Implement BETWEEN qualifier(Bruce)
Implement IN qualifier(Bruce)
libpq has PQgetisnull()(Bruce)
diff --git a/src/backend/optimizer/geqo/geqo_erx.c
b/src/backend/optimizer/geqo/geqo_erx.c
--- a/src/backend/optimizer/geqo/geqo_erx.c
+++ b/src/backend/optimizer/geqo/geqo_erx.c
@@ -458,7 +458,7 @@ edge_failure(PlannerInfo *root, Gene *ge
if (edge_table[i].unused_edges >= 0)
return (Gene) i;
- elog(LOG, "no edge found via looking for the last ununsed
point");
+ elog(LOG, "no edge found via looking for the last unused
point");
}
diff --git a/src/backend/port/dynloader/linux.c
b/src/backend/port/dynloader/linux.c
--- a/src/backend/port/dynloader/linux.c
+++ b/src/backend/port/dynloader/linux.c
@@ -124,7 +124,7 @@ char *
pg
Re: [HACKERS] Possible spelling fixes
Heikki wrote: > I pushed most of these. Except for the below: > optimisation -> optimization et al. > Most of our code is written with the American spelling, > but the British spelling isn't wrong, > so I don't want to go around changing them all. Sure As you'll see, my approach is to aim for consistency. If you used en-GB 99% of the time, I'd have offered a change to enforce that. I have a personal preference, but there's no obligation, and I understand the potential costs churn entails (I see you backported to branches). > NUL-terminated -> NULL-terminated > When we're talking about NUL-terminated strings, > NUL refers to the NUL ASCII character. NULL usually refers to a NULL pointer. This wasn't even in my original set (i.e. The dictionary I'm using didn't consider NUL to be misspelled). I ran across it while splitting comments out per Andres and figured I'd offer it as well. > We're probably not consistent about this, Hrm, I was going to say "that's correct, you aren't", but rereading, I realize that I'd have to review every instance in order to prove to myself that statement. I could make the weaker argument that "nul-terminated" should be changed to either NUL-terminated or null-terminated . My general approach is to only make changes when I can detect an inconsistency. And I generally tend toward "majority rule". Here, I think the corpus has 4 spellings, and it sounds like it should only have two, but probably (NUL- and null-) not the two I picked (NULL- and null-). > but in this context, NUL-terminated isn't wrong, so let's leave them as they > are. But that's OK. My goal in posting these is to encourage people to consider their code. >> Ooops -> Oops > "Oops" is more idiomatic, but this doesn't really seem worth changing. Technically oops is in dictionaries whereas the other isn't, but I understood the intent. > Maybe "Ooops" indicates a slightly bigger mistake than "oops" :-) That seemed like the intent. It's certainly not unreasonable to retain it. It's also why I generally offer a queue, so people can reject families of changes. >> re-entrancy -> reentrancy > Googling around, I can see both spellings being used. Both are used, but reentrancy is in most dictionaries (and encyclopedias) and is the form that's used in instruction (certainly it was when I studied in university, and it isn't likely to regress). It's akin to email vs e-mail. Once the dashless form becomes accepted (within a domain [1]), it's the correct form, and the other was merely transitional. > "Re-entrancy" actually feels more natural to me, although I'm not sure which > is more correct. > Let's leave them as they are. Sure >> passthru -> passthrough > "Passthrough" is clearly the correct spelling (or "pass-through"?), The former is also present in the codebase. (I didn't look for the latter, for the same reason as the previous note.) > but "passthru" seems OK in the context, as an informal shorthand. My goal is consistency. If you always spell a concept a single way, then grepping for that concept is easier and more reliable. I personally recognize quite a few flavors, because they're usable for talking to Coverity / Purify. >> - * Temporay we use TSLexeme.flags for inner use... >> + * Temporary we use TSLexeme.flags for inner use... > Looking at the code real quick, I couldn't understand the original meaning of this. Is it: > * DT_USEASIS is a temporary value we use for something. For what? > * DT_USEASIS is used temporarily for something. > Does this mean, "temporarily" until we get around to write the code > differently, or does > it happen temporarily at runtime, or what? > Just fixing the typo doesn't help much here, > and I'm not sure if it should be "temporary" or "temporarily" anyway. Apparently I didn't look at this one much at all. I believe temporarily is the intended word (fwiw, I originally mis-corrected directly as directory, that I did spot before submitting). And probably as a runtime concept. But I'm not volunteering to fix all comments in the project ;-). After spelling fixes, I'm more likely to try actual bugs / usability issues. I have a specific bug which bit me, but fixing that would require more effort than a spelling pass and more cooperation. I tend to do a spelling pass to determine if the more expensive activity is viable. So far, the project is welcoming :-) so, perhaps I'll manage to write the real fix... > I wasn't sure if this changes the meaning of the comment slightly. > An "UPDATE" in all-caps refers to an UPDATE statement, > is that what's meant here? Or just updating a tuple, > i.e. should this rather be "skip updating of the tuple" or "skip update of > tuple"? I'm not certain. I do understand that capital UPDATE is special. This one people more familiar with the project will have to resolve. Fwiw, if it's the former, you could omit the "of". > This "postsql" refers to the SQL dialect of PostgreSQL, I had t
Re: [HACKERS] Possible spelling fixes
It's now split more or less to your suggestion:
https://github.com/jsoref/postgres/commits/spelling
diff --git a/configure b/configure
--- a/configure
+++ b/configure
@@ -7088,7 +7088,7 @@ test -z "$INSTALL_SCRIPT" && INSTALL_SCR
test -z "$INSTALL_DATA" && INSTALL_DATA='${INSTALL} -m 644'
# When Autoconf chooses install-sh as install program it tries to generate
-# a relative path to it in each makefile where it subsitutes it. This clashes
+# a relative path to it in each makefile where it substitutes it. This clashes
# with our Makefile.global concept. This workaround helps.
case $INSTALL in
*install-sh*) install_bin='';;
@@ -7232,7 +7232,7 @@ fi
$as_echo "$MKDIR_P" >&6; }
# When Autoconf chooses install-sh as mkdir -p program it tries to generate
-# a relative path to it in each makefile where it subsitutes it. This clashes
+# a relative path to it in each makefile where it substitutes it. This clashes
# with our Makefile.global concept. This workaround helps.
case $MKDIR_P in
*install-sh*) MKDIR_P='\${SHELL} \${top_srcdir}/config/install-sh -c -d';;
diff --git a/configure.in b/configure.in
--- a/configure.in
+++ b/configure.in
@@ -887,7 +887,7 @@ fi
AC_PROG_INSTALL
# When Autoconf chooses install-sh as install program it tries to generate
-# a relative path to it in each makefile where it subsitutes it. This clashes
+# a relative path to it in each makefile where it substitutes it. This clashes
# with our Makefile.global concept. This workaround helps.
case $INSTALL in
*install-sh*) install_bin='';;
@@ -900,7 +900,7 @@ AC_PROG_LN_S
AC_PROG_AWK
AC_PROG_MKDIR_P
# When Autoconf chooses install-sh as mkdir -p program it tries to generate
-# a relative path to it in each makefile where it subsitutes it. This clashes
+# a relative path to it in each makefile where it substitutes it. This clashes
# with our Makefile.global concept. This workaround helps.
case $MKDIR_P in
*install-sh*) MKDIR_P='\${SHELL} \${top_srcdir}/config/install-sh -c -d';;
diff --git a/contrib/bloom/blvacuum.c b/contrib/bloom/blvacuum.c
--- a/contrib/bloom/blvacuum.c
+++ b/contrib/bloom/blvacuum.c
@@ -51,7 +51,7 @@ blbulkdelete(IndexVacuumInfo *info, Inde
initBloomState(&state, index);
/*
-* Interate over the pages. We don't care about concurrently added
pages,
+* Iterate over the pages. We don't care about concurrently added pages,
* they can't contain tuples to delete.
*/
npages = RelationGetNumberOfBlocks(index);
diff --git a/contrib/cube/sql/cube.sql b/contrib/cube/sql/cube.sql
--- a/contrib/cube/sql/cube.sql
+++ b/contrib/cube/sql/cube.sql
@@ -256,7 +256,7 @@ SELECT cube_dim('(0,0,0)'::cube);
SELECT cube_dim('(42,42,42),(42,42,42)'::cube);
SELECT cube_dim('(4,8,15,16,23),(4,8,15,16,23)'::cube);
--- Test of cube_ll_coord function (retrieves LL coodinate values)
+-- Test of cube_ll_coord function (retrieves LL coordinate values)
--
SELECT cube_ll_coord('(-1,1),(2,-2)'::cube, 1);
SELECT cube_ll_coord('(-1,1),(2,-2)'::cube, 2);
@@ -268,7 +268,7 @@ SELECT cube_ll_coord('(42,137)'::cube, 1
SELECT cube_ll_coord('(42,137)'::cube, 2);
SELECT cube_ll_coord('(42,137)'::cube, 3);
--- Test of cube_ur_coord function (retrieves UR coodinate values)
+-- Test of cube_ur_coord function (retrieves UR coordinate values)
--
SELECT cube_ur_coord('(-1,1),(2,-2)'::cube, 1);
SELECT cube_ur_coord('(-1,1),(2,-2)'::cube, 2);
diff --git a/contrib/earthdistance/earthdistance--1.1.sql
b/contrib/earthdistance/earthdistance--1.1.sql
--- a/contrib/earthdistance/earthdistance--1.1.sql
+++ b/contrib/earthdistance/earthdistance--1.1.sql
@@ -11,7 +11,7 @@ CREATE FUNCTION earth() RETURNS float8
LANGUAGE SQL IMMUTABLE PARALLEL SAFE
AS 'SELECT ''6378168''::float8';
--- Astromers may want to change the earth function so that distances will be
+-- Astronomers may want to change the earth function so that distances will be
-- returned in degrees. To do this comment out the above definition and
-- uncomment the one below. Note that doing this will break the regression
-- tests.
diff --git a/contrib/isn/ISSN.h b/contrib/isn/ISSN.h
--- a/contrib/isn/ISSN.h
+++ b/contrib/isn/ISSN.h
@@ -23,7 +23,7 @@
* Product 9 + 21 + 7 + 3 + 1 + 12 + 4 + 24 + 7 + 15 + 0 + 0 = 103
* 103 / 10 = 10 remainder 3
* Check digit 10 - 3 = 7
- * => 977-1144875-00-7 ?? <- suplemental number (number of the week, month,
etc.)
+ * => 977-1144875-00-7 ?? <- supplemental number (number of the week, month,
etc.)
* ^^ 00 for non-daily publications (01=Monday,
02=Tuesday, ...)
*
* The hyphenation is always in after the four digits of the ISSN code.
diff --git a/contrib/isn/isn.c b/contrib/isn/isn.c
--- a/contrib/isn/isn.c
+++ b/contrib/isn/isn.c
@@ -160,7 +160,7 @@ dehyphenate(char *bufO, char *bufI)
* into bufO using the given hyphenation range
TABLE.
*
[HACKERS] Possible spelling fixes
Hi. I'm going through project-by-project offering spelling fixes. I have read your submission suggestions [1][2] and am choosing to disregard them. Could someone please review the changes I have [3] and suggest a series of commits that this project might like? It is quite likely that someone will tell me that there are certain types of changes that you don't want. That's fine, I'm happy to drop changes. It's possible that I've chosen the wrong spelling correction or made a mistake in my attempts, while the process of identifying errant words is somewhat automated, the correction process is a mix of me and some scripts, and neither of us are perfect. The patch queue is currently sorted alphabetically, but it contains at least: * branding corrections -- some of which may be wrong * regional English -- I understand there's a discussion about this, I can easily drop them, or you can choose to take them separately * changelog changes -- I could understand not wanting to change the changelog * local variable changes * API changes -- hopefully the changes to functions/enums are within private space and thus not a problem, but if you consider them to be public, dropping them isn't a big deal * changes to tests -- it wouldn't shock me if I messed some of these up. Again, everything can be split/dropped/fixed if requested (within limits -- I refuse to submit 100 emails with patches) * changes to comments, lots of comments Most changes are just changes to comments, and I'd hope it wouldn't be terribly hard for them to be accepted. A complete diff would be roughly 130k. I've recently tried to submit a similarly sized patch to another project and it was stuck in moderation (typically mailing lists limit attachments to around 40k). I do not expect anyone to be able to do anything remotely useful with such a patch. It is certainly possible for someone else to generate such a file if they really like reading a diff in that manner. For reference, I've split my proposal into 172 changes. Each change should represent a single word/concept which has been misspelled (possibly in multiple different manners). I understand that people don't care about diffstats (I certainly don't), but for reference collectively this encompasses 175 files and 305 lines. Note that I've intentionally excluded certain files (by removing them in prior commits), it's possible that changes would be needed to be made to some of those files in order for tests to pass. If you have an automated system for running tests (typically Travis or Appveyor), I'm happy to submit changes to them before offering changes to a list. But I couldn't find anything of the sort. [1] https://wiki.postgresql.org/wiki/Submitting_a_Patch [2] http://petereisentraut.blogspot.ca/2009/09/how-to-submit-patch-by-email.html [3] https://github.com/jsoref/postgres/compare/ignore...jsoref:spelling?expand=1 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Retiring from the Core Team
Hackers: You will have noticed that I haven't been very active for the past year. My new work on Linux containers and Kubernetes has been even more absorbing than I anticipated, and I just haven't had a lot of time for PostgreSQL work. For that reason, as of today, I am stepping down from the PostgreSQL Core Team. I joined the PostgreSQL Core Team in 2003. I decided to take on project advocacy, with the goal of making PostgreSQL one of the top three databases in the world. Thanks to the many contributions by both advocacy volunteers and developers -- as well as the efforts by companies like EnterpriseDB and Heroku -- we've achieved that goal. Along the way, we proved that community ownership of an OSS project can compete with, and ultimately outlast, venture-funded startups. Now we need new leadership who can take PostgreSQL to the next phase of world domination. So I am joining Vadim, Jan, Thomas, and Marc in clearing the way for others. I'll still be around and still contributing to PostgreSQL in various ways, mostly around running the database in container clouds. It'll take a while for me to hand off all of my PR responsibilities for the project (assuming that I ever hand all of them off). It's been a long, fun ride, and I'm proud of the PostgreSQL we have today: both the database, and the community. Thank you for sharing it with me. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RustgreSQL
On 01/09/2017 05:54 PM, Joel Jacobson wrote: > On Mon, Jan 9, 2017 at 3:22 PM, Jim Nasby wrote: >> I do wonder if there are parts of the codebase that would be much better >> suited to a language other than C, and could reasonably be ported. >> Especially if that could be done in such a way that the net result is still >> C code so we're not adding dependencies to non developers (similar to >> bison). >> >> Extensions are a step in that direction, but they're ultimately not core >> Postgres (which is a different issue). > > I think this is a great idea! > > That way the amount of C code could be reduced over time, > while safely extending the official version with new functionality on > the surface, > without increasing the amount of C code. Even if you don't ever end up touching core Postgres, being able to write extensions in languages other than C (like, full-featured extensions) would be its own benefit. Why not start there? That is, assuming that Joel has gobs of time to work on this? For that matter, I know that Jeff Davis is quite fond of Rust. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for changes to recovery.conf API
On 01/03/2017 08:47 AM, Robert Haas wrote: > On Tue, Jan 3, 2017 at 11:21 AM, Simon Riggs wrote: >> On 3 January 2017 at 15:50, Robert Haas wrote: >>> On Sun, Jan 1, 2017 at 4:14 PM, Simon Riggs wrote: >>>> Trying to fit recovery targets into one parameter was really not >>>> feasible, though I tried. >>> >>> What was the problem? >> >> There are 5 different parameters that affect the recovery target, 3 of >> which are optional. That is much more complex than the two compulsory >> parameters suggested when the proposal was made and in my view tips >> the balance over the edge into pointlessness. Michael's suggestion for >> 2 parameters works well for this case. > > I still think merging recovery_target_type and recovery_target_value > into a single parameter could make sense, never mind the other three. > But I don't really want to argue about it any more. > Either solution works for me. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Migration Docs WAS: Proposal for changes to recovery.conf API
On 12/19/2016 01:29 PM, Peter Eisentraut wrote: > On 12/16/16 8:52 PM, Robert Haas wrote: >> > If the explanation is just a few sentences long, I see no reason not >> > to include it in the release notes. > As far as I can tell from the latest posted patch, the upgrade > instructions are > > - To trigger recovery, create an empty file recovery.trigger instead of > recovery.conf. > > - All parameters formerly in recovery.conf are now regular > postgresql.conf parameters. For backward compatibility, recovery.conf > is read after postgresql.conf, but the parameters can now be put into > postgresql.conf if desired. Aren't we changing how some of the parameters work? > > Some of that might be subject to patch review, but it's probably not > going to be much longer than that. So I think that will fit well into > the usual release notes section. Changed the subject line of this thread because people are becoming confused about the new topic. I'm not talking about *just* the recovery.conf changes. We're making a lot of changes to 10 which will require user action, and there may be more before 10 is baked. For example, dealing with the version numbering change. I started a list of the things we're breaking for 10, but I don't have it with me at the moment. There's more than 3 things on it. And then there's docs for stuff which isn't *required* by upgrading, but would be a good idea. For example, we'll eventually want a doc on how to migrate old-style partitioned tables to new-style partitioned tables. In any case, Peter's response shows *exactly* why I don't want to put this documentation into the release notes: because people are going to complain it's too long and want to truncate it. Writing the docs will be hard enough; if I (or anyone else) has to argue about whether or not they're too long, I'm just going to drop the patch and walk away. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for changes to recovery.conf API
On 12/15/2016 12:54 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Thu, Dec 15, 2016 at 1:11 AM, Bruce Momjian wrote: >>> You are saying this is more massive than any other change we have made >>> in the past? In general, what need to be documented? > >> I don't necessarily think it's because it's more massive than any chance we >> have made before. I think it's more that this is something that we probably >> should've had before, and just didn't. > >> Right now we basically have a bulletpoint list of things that are new, with >> a section about things that are incompatible. Having an actual section >> with more detailed descriptions of how to handle these changes would >> definitely be a win. it shouldn't *just* be for these changes of course, it >> should be for any other changes that are large enough to benefit from more >> than a oneliner about the fact that they've changed. > > Yeah, it seems to me that where this is ending up is "we may need to > write more in the compatibility entries than we have in the past". > I don't see any problem with that, particularly if someone other than > Bruce or me is volunteering to write it ;-) I'm up for writing it (with help from feature owners), provided that I don't have to spend time arguing that it's not too long, or that I should put it somewhere different. So can we settle the "where" question first? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for changes to recovery.conf API
On 12/14/2016 08:06 AM, Bruce Momjian wrote: > On Fri, Dec 9, 2016 at 09:46:44AM +0900, Michael Paquier wrote: >>>> My own take on it is that the release notes are already a massive >>>> amount of work, and putting duplicative material in a bunch of other >>>> places isn't going to make things better, it'll just increase the >>>> maintenance burden. >>> >>> This would mean adding literally pages of material to the release notes. >>> In the past, folks have been very negative on anything which would make >>> the release notes longer. Are you sure? >> >> As that's a per-version information, that seems adapted to me. There >> could be as well in the release notes a link to the portion of the >> docs holding this manual. Definitely this should be self-contained in >> the docs, and not mention the wiki. My 2c. > > Yes, that is the usual approach. > So where in the docs should these go, then? We don't (currently) have a place for this kind of doc. Appendices? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for changes to recovery.conf API
On 12/08/2016 04:16 PM, Tom Lane wrote: > Josh Berkus writes: >> On 12/01/2016 05:58 PM, Magnus Hagander wrote: >>> And in fairness, having such a "guide to changes" chapter in each >>> release probably *would* be a good idea. But it would take resources to >>> make that. The release notes are good, but having a more hand-holding >>> version explaining incompatible changes in "regular sentences" would >>> probably be quite useful to users. > >> We will have enough major changes in 10.0 to warrant writing one of >> these. Maybe not as part of the official docs, but as a set of wiki >> pages or similar. > > Seems to me this is exactly the release notes' turf. If you think the > release notes aren't clear enough, step right up and help improve them. > > My own take on it is that the release notes are already a massive > amount of work, and putting duplicative material in a bunch of other > places isn't going to make things better, it'll just increase the > maintenance burden. This would mean adding literally pages of material to the release notes. In the past, folks have been very negative on anything which would make the release notes longer. Are you sure? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for changes to recovery.conf API
On 12/01/2016 05:58 PM, Magnus Hagander wrote: > >> * Add docs: "Guide to changes in recovery.conf in 10.0" > > > > Hmm, we don't usually write the docs in terms of how things are > > different from a previous version. Might seem strange in 5 years. > > Not sure what's best, here. > > A good chunk in the release notes would make sense as well? > > > It would. > > And in fairness, having such a "guide to changes" chapter in each > release probably *would* be a good idea. But it would take resources to > make that. The release notes are good, but having a more hand-holding > version explaining incompatible changes in "regular sentences" would > probably be quite useful to users. We will have enough major changes in 10.0 to warrant writing one of these. Maybe not as part of the official docs, but as a set of wiki pages or similar. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for changes to recovery.conf API
On 12/04/2016 07:21 PM, Michael Paquier wrote: > On Mon, Dec 5, 2016 at 11:34 AM, Haribabu Kommi > wrote: >> As there was a feedback from others related to the patch and doesn't find >> any >> update from author. >> >> Closed in 2016-11 commitfest with "returned with feedback" status. >> Please feel free to update the status once you submit the updated patch or >> if the current status doesn't reflect the actual status of the patch. > > Having a consensus here is already a great step forward. I am sure > that a new version will be sent for the next CF. > Please let's make sure this gets done for 10. We're planning on breaking a lot of config things in 10, so it would be MUCH easier on users if we do the recovery.conf changes in that version as well. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_hba_file_settings view patch
On 10/26/2016 12:24 PM, Tom Lane wrote: > Robert Haas writes: >> FWIW, I'm -1 on using JSON here. I don't believe that we should start >> using JSON all over the place just because we can. If we do that, >> we'll end up with a mishmash of styles, and maybe look silly when JSON >> is replaced by the new and much better SDGJHSDR format. > > I concur. JSON isn't a core datatype and I don't want to see it treated > as one. We should redesign this view so that it doesn't rely on anything > more advanced than arrays. Huh? Sure it is. Ships in PostgreSQL-core. I mean, I'm not particularly in favor of using JSON for this (arrays seem OK), but that seems like an invalid reason not to. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default setting for autovacuum_freeze_max_age
On 10/21/2016 10:29 AM, Robert Haas wrote: > On Fri, Oct 21, 2016 at 1:17 PM, Josh Berkus wrote: >> Particularly, with 9.6's freeze map, point (2) is even stronger reason >> to *lower* autovacuum_max_freeze_age. Since there's little duplicate >> work in a freeze scan, a lot of users will find that frequent freezing >> benefits them a lot ... > > That's a very good point, although I hope that vacuum is mostly being > triggered by vacuum_freeze_table_age rather than > autovacuum_freeze_max_age. Well, depends on the nature of writes to the table. For insert-mostly tables, vacuum_freeze_table_age is pretty much never triggered. Isn't there a patch for that somewhere? > > On Bruce's original question, there is an answer written into our > documentation: "Vacuum also allows removal of old files from the > pg_clog subdirectory, which is why the default is a relatively low 200 > million transactions." Point. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Proposal for changes to recovery.conf API
On 09/28/2016 10:13 AM, Robert Haas wrote: > On Tue, Sep 6, 2016 at 10:11 AM, David Steele wrote: >> On 9/6/16 8:07 AM, Robert Haas wrote: >>> On Wed, Aug 31, 2016 at 9:45 PM, Simon Riggs >>> wrote: >>>> Related cleanup >>>> * Promotion signal file is now called "promote.trigger" rather than >>>> just "promote" >>>> * Remove user configurable "trigger_file" mechanism - use >>>> "promote.trigger" for all cases >>> >>> >>> I'm in favor of this. I don't think that it's very hard for authors >>> of backup tools to adapt to this new world, and I don't see that >>> allowing configurability here does anything other than create more >>> cases to worry about. >> >> +1 from a backup tool author. > > It's time to wrap up this CommitFest, and this thread doesn't seem to > contain anything that looks like a committable patch. So, I'm marking > this "Returned with Feedback". I hope that the fact that there's been > no discussion for the last three weeks doesn't mean this effort is > dead; I would like very much to see it move forward. Has this gone anywhere? Given that we're in "break all the things" mode for PostgreSQL 10, it would be the ideal time to consolidate recovery.conf with pg.conf. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Default setting for autovacuum_freeze_max_age
On 10/21/2016 07:44 AM, Tom Lane wrote: > Bruce Momjian writes: >> Why is autovacuum_freeze_max_age's default set to 200 million, rather >> than something like 2 billion? It seems 2 billion is half way to >> wrap-around and would be a better default. Right now, the default seems >> to freeze 10x more often than it has to. > > Please see the archives. I do not remember the reasoning, but there > was some, and you need to justify why it was wrong not just assert > that you think it's silly. IIRC, there were a couple reasons (and I think they're still good reasons, which is why I haven't asked to change the default): 1. By setting it to 10% of the max space, we give users plenty of room to raise the number if they need to without getting into crisis territory. 2. Raising this threshold isn't an unalloyed good. The longer you wait to freeze, the more work you'll need to do when autovac freeze rolls around. There's actually situations where you want to make this threshold *lower*, although generally scheduled manual vacuum freezes serve that. Particularly, with 9.6's freeze map, point (2) is even stronger reason to *lower* autovacuum_max_freeze_age. Since there's little duplicate work in a freeze scan, a lot of users will find that frequent freezing benefits them a lot ... especially if they can take advantage of index-only scans. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Disable autovacuum guc?
On 10/20/2016 06:34 AM, Joshua D. Drake wrote: > On 10/19/2016 07:22 PM, Josh Berkus wrote: >> On 10/19/2016 06:27 PM, Joshua D. Drake wrote: >>> Hello, >>> >>> After all these years, we are still regularly running into people who >>> say, "performance was bad so we disabled autovacuum". I am not talking >>> about once in a while, it is often. I would like us to consider removing >>> the autovacuum option. Here are a few reasons: >>> >>> 1. It does not hurt anyone >>> 2. It removes a foot gun >>> 3. Autovacuum is *not* optional, we shouldn't let it be >>> 4. People could still disable it at the table level for those tables >>> that do fall into the small window of, no maintenance is o.k. >>> 5. People would still have the ability to decrease the max_workers to 1 >>> (although I could argue about that too). >> >> People who run data warehouses where all of the data comes in as batch >> loads regularly disable autovacuum, and should do so. For the DW/batch >> load use-case, it makes far more sense to do batch loads interspersed >> with ANALYZEs and VACUUMS of loaded/updated tables. > > Hrm, true although that is by far a minority of our users. What if we > made it so we disabled the autovacuum guc but made it so you could > disable autovacuum per database (ALTER DATABASE SET or something such > thing?). Well, that wouldn't fix the problem; people would just disable it per database, even if it was a bad idea. If I can't get rid of vacuum_defer_cleanup_age, you're not going to be able to get rid of autovacuum. Now, if you want to "fix" this issue, one thing which would help a lot is making it possible to disable/enable Autovacuum on one table without exclusive locking (for the ALTER statement), and create a how-to doc somewhere which explains how and why to disable autovac per-table. Most of our users don't know that it's even possible to adjust this per-table. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Disable autovacuum guc?
On 10/19/2016 06:27 PM, Joshua D. Drake wrote: > Hello, > > After all these years, we are still regularly running into people who > say, "performance was bad so we disabled autovacuum". I am not talking > about once in a while, it is often. I would like us to consider removing > the autovacuum option. Here are a few reasons: > > 1. It does not hurt anyone > 2. It removes a foot gun > 3. Autovacuum is *not* optional, we shouldn't let it be > 4. People could still disable it at the table level for those tables > that do fall into the small window of, no maintenance is o.k. > 5. People would still have the ability to decrease the max_workers to 1 > (although I could argue about that too). People who run data warehouses where all of the data comes in as batch loads regularly disable autovacuum, and should do so. For the DW/batch load use-case, it makes far more sense to do batch loads interspersed with ANALYZEs and VACUUMS of loaded/updated tables. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On 10/19/2016 09:59 AM, Bruce Momjian wrote: > On Wed, Oct 19, 2016 at 09:00:06AM -0400, Robert Haas wrote: >> On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjian wrote: >>> On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote: >>>> Actually, I think vacuum_defer_cleanup_age is, and always has been, an >>>> ugly hack. But for some people it may be the ugly hack that is >>>> letting them continue to use PostgreSQL. >>> >>> I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby >>> servers --- it cancels transactions rather than delaying cleanup. >> >> I think it's the opposite, isn't it? vacuum_defer_cleanup_age >> prevents cancellations. > > Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms > of xids, that a standby query can run before cancel, like > old_snapshot_threshold, no? After that, we can cancel standby queries. > I see hot_standby_feedback as our current behavior on the master where > we never cancel standby queries. > > To me, hot_standby_feedback extends no-cleanup-no-cancel from the > standby to the master, while vacuum_defer_cleanup_age behaves like > old_snapshot_threshold in that it causes cancel for long-running > queries. See Andres' response on this thread. He's already covered why the setting is still useful, but why we might want to remove it anyway. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On 10/18/2016 01:37 PM, Andres Freund wrote: > Hi, > > On 2016-10-09 21:51:07 -0700, Josh Berkus wrote: >> Given that hot_standby_feedback is pretty bulletproof now, and a lot of >> the work in reducing replay conflicts, I think the utility of >> vacuum_defer_cleanup_age is at an end. I really meant so submit a patch >> to remove it to 9.6, but it got away from me. > > HS feedback doesn't e.g. work well with delayed and/or archived replay, > whereas defer_cleanup does. Oh, point! See, that's why I polled, I knew there was something I was forgetting about. > On the other hand, removing it would make some of the reasoning around > GetOldestXmin() a bit easier. Enough to make it worth breaking the above? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On 10/18/2016 01:28 PM, Robert Haas wrote: > On Tue, Oct 18, 2016 at 4:18 PM, Josh Berkus wrote: >> On 10/12/2016 05:00 PM, Robert Haas wrote: >>> On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus wrote: >>>> Given that hot_standby_feedback is pretty bulletproof now, and a lot of >>>> the work in reducing replay conflicts, I think the utility of >>>> vacuum_defer_cleanup_age is at an end. I really meant so submit a patch >>>> to remove it to 9.6, but it got away from me. >>>> >>>> Any objections to removing the option in 10? >>> >>> I'm not sure I see the point. >> >> Redusing the number of configuration variables is an a-priori good. In >> aggregate, the more knobs we have, the harder it is to learn how to >> admin Postgres. Therefore any time a config variable becomes obsolete, >> we should remove it. > > Meh. I agree that more configuration knobs makes it harder to learn > to configure the system, but we've got enough of them that removing > exactly one isn't going to make a material difference. Against that, > if you are wrong about it being obsolete and there are actually people > relying on it heavily, those people will be very sad if we remove it, > and unless they read this mailing list, we probably won't find out > until it's too late. Based on that argument, we would never be able to remove any configuration parameter ever. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On 10/12/2016 05:00 PM, Robert Haas wrote: > On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus wrote: >> Given that hot_standby_feedback is pretty bulletproof now, and a lot of >> the work in reducing replay conflicts, I think the utility of >> vacuum_defer_cleanup_age is at an end. I really meant so submit a patch >> to remove it to 9.6, but it got away from me. >> >> Any objections to removing the option in 10? > > I'm not sure I see the point. Redusing the number of configuration variables is an a-priori good. In aggregate, the more knobs we have, the harder it is to learn how to admin Postgres. Therefore any time a config variable becomes obsolete, we should remove it. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Remove vacuum_defer_cleanup_age
Folks, Given that hot_standby_feedback is pretty bulletproof now, and a lot of the work in reducing replay conflicts, I think the utility of vacuum_defer_cleanup_age is at an end. I really meant so submit a patch to remove it to 9.6, but it got away from me. Any objections to removing the option in 10? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Congrats on the on-time release!
Hackers, I wanted to congratulate everyone involved (and it's a long list of people) in having our first on-schedule major release since 9.3. Especially the RMT, which did a lot to make that happen. Getting the release train to run on time is a major accomplishment, and will help both development and adoption. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Quorum commit for multiple synchronous replication.
On 08/29/2016 06:52 AM, Fujii Masao wrote: > Also I like the following Simon's idea. > > https://www.postgresql.org/message-id/canp8+jlhfbvv_pw6grasnupw+bdk5dctu7gwpnap-+-zwvk...@mail.gmail.com > --- > * first k (n1, n2, n3) – does the same as k (n1, n2, n3) does now > * any k (n1, n2, n3) – would release waiters as soon as we have the > responses from k out of N standbys. “any k” would be faster, so is > desirable for performance and resilience What are we going to do for backwards compatibility, here? So, here's the dilemma: If we want to keep backwards compatibility with 9.6, then: "k (n1, n2, n3)" == "first k (n1, n2, n3)" However, "first k" is not what most users will want, most of the time; users of version 13, years from now, will be getting constantly confused by "first k" behavior when they wanted quorum. So the sensible default would be: "k (n1, n2, n3)" == "any k (n1, n2, n3)" ... however, that will break backwards compatibility. Thoughts? My $0.02 is that we break backwards compat somehow and document the heck out of it. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] increasing the default WAL segment size
On 08/25/2016 01:12 PM, Robert Haas wrote: >> I agree that #4 is best. I'm not sure it's worth the cost. I'm not worried >> > at all about the risk of master/slave sync thing, per previous statement. >> > But if it does have performance implications, per Andres suggestion, then >> > making it configurable at initdb time probably comes with a cost that's not >> > worth paying. > At this point it's hard to judge, because we don't have any idea what > the cost might be. I guess if we want to pursue this approach, > somebody will have to code it up and benchmark it. But what I'm > inclined to do for starters is put together a patch to go from 16MB -> > 64MB. Committing that early this cycle will give us time to > reconsider if that turns out to be painful for reasons we haven't > thought of yet. And give tool authors time to make adjustments, if > any are needed. The one thing I'd be worried about with the increase in size is folks using PostgreSQL for very small databases. If your database is only 30MB or so in size, the increase in size of the WAL will be pretty significant (+144MB for the base 3 WAL segments). I'm not sure this is a real problem which users will notice (in today's scales, 144MB ain't much), but if it turns out to be, it would be nice to have a way to switch it back *just for them* without recompiling. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PSA: Systemd will kill PostgreSQL
On 08/15/2016 05:18 PM, Tom Lane wrote: > Josh Berkus writes: >> On 08/15/2016 02:43 PM, Tom Lane wrote: >>> Last I heard, there's an exclusion for "system" accounts, so an >>> installation that's using the Fedora-provided pgsql account isn't >>> going to have a problem. It's homebrew installs running under >>> ordinary-user accounts that are at risk. > >> Presumably people just need to add the system account tag to the unit >> file, no? > > Well, yeah, it's easy to fix once you know you need to do so. The > complaint is basically that out-of-the-box, it's broken, and it's > not very clear what was gained by breaking it. You're welcome to argue with Lennart about that. I'm not personally supporting the feature, I just don't think it's that hard to work around. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PSA: Systemd will kill PostgreSQL
On 08/15/2016 02:43 PM, Tom Lane wrote: > Josh Berkus writes: >> On 07/10/2016 10:56 AM, Joshua D. Drake wrote: >>> tl;dr; Systemd 212 defaults to remove all IPC (including SYSV memory) >>> when a user "fully" logs out. > >> That looks like it was under discussion in April, though. Do we have >> confirmation it was never fixed? I'm not seeing systemd killing >> Postgres under Fedora24. > > Last I heard, there's an exclusion for "system" accounts, so an > installation that's using the Fedora-provided pgsql account isn't > going to have a problem. It's homebrew installs running under > ordinary-user accounts that are at risk. Presumably people just need to add the system account tag to the unit file, no? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PSA: Systemd will kill PostgreSQL
On 07/10/2016 10:56 AM, Joshua D. Drake wrote: > Hackers, > > This just came across my twitter feed: > > https://lists.freedesktop.org/archives/systemd-devel/2014-April/018373.html > > tl;dr; Systemd 212 defaults to remove all IPC (including SYSV memory) > when a user "fully" logs out. That looks like it was under discussion in April, though. Do we have confirmation it was never fixed? I'm not seeing systemd killing Postgres under Fedora24. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why we lost Uber as a user
On 07/28/2016 03:58 AM, Geoff Winkless wrote: > On 27 July 2016 at 17:04, Bruce Momjian <mailto:br...@momjian.us>>wrote: > > Well, their big complaint about binary replication is that a bug can > spread from a master to all slaves, which doesn't happen with statement > level replication. > > > > I'm not sure that that makes sense to me. If there's a database bug > that occurs when you run a statement on the master, it seems there's a > decent chance that that same bug is going to occur when you run the same > statement on the slave. > > Obviously it depends on the type of bug and how identical the slave is, > but statement-level replication certainly doesn't preclude such a bug > from propagating. That's correct, which is why I ignored that part of their post. However, we did have issues for a couple of years where replication accuracy was poorly tested, and did have several bugs associated with that. It's not surprising that a few major users got hit hard by those bugs and decided to switch. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why we lost Uber as a user
On 07/26/2016 08:45 PM, Robert Haas wrote: > That's why I found Josh's restatement useful - I am assuming without > proof that his restatement is accurate FWIW, my restatement was based on some other sites rather than Uber. Including folks who didn't abandon Postgres. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why we lost Uber as a user
On 07/26/2016 03:07 PM, Tom Lane wrote: > Josh Berkus writes: >> That's a recipe for runaway table bloat; VACUUM can't do much because >> there's always some minutes-old transaction hanging around (and SNAPSHOT >> TOO OLD doesn't really help, we're talking about minutes here), and >> because of all of the indexes HOT isn't effective. > > Hm, I'm not following why this is a disaster. OK, you have circa 100% > turnover of the table in the lifespan of the slower transactions, but I'd > still expect vacuuming to be able to hold the bloat to some small integer > multiple of the minimum possible table size. Not in practice. Don't forget that you also have bloat of the indexes as well. I encountered multiple cases of this particular failure case, and often bloat ended up at something like 100X of the clean table/index size, with no stable size (that is, it always kept growing). This was the original impetus for wanting REINDEX CONCURRENTLY, but really that's kind of a workaround. (And if the table is small, > that's still small.) I suppose really long transactions (pg_dump?) could > be pretty disastrous, but there are ways around that, like doing pg_dump > on a slave. You'd need a dedicated slave for the pg_dump, otherwise you'd hit query cancel. > Or in short, this seems like an annoyance, not a time-for-a-new-database > kind of problem. It's considerably more than an annoyance for the people who suffer from it; for some databases I dealt with, this one issue was responsible for 80% of administrative overhead (cron jobs, reindexing, timeouts ...). But no, it's not a database-switcher *by itself*. But is is a chronic, and serious, problem. I don't have even a suggestion of a real solution for it without breaking something else, though. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why we lost Uber as a user
On 07/26/2016 01:53 PM, Josh Berkus wrote: > The write amplification issue, and its correllary in VACUUM, certainly > continues to plague some users, and doesn't have any easy solutions. To explain this in concrete terms, which the blog post does not: 1. Create a small table, but one with enough rows that indexes make sense (say 50,000 rows). 2. Make this table used in JOINs all over your database. 3. To support these JOINs, index most of the columns in the small table. 4. Now, update that small table 500 times per second. That's a recipe for runaway table bloat; VACUUM can't do much because there's always some minutes-old transaction hanging around (and SNAPSHOT TOO OLD doesn't really help, we're talking about minutes here), and because of all of the indexes HOT isn't effective. Removing the indexes is equally painful because it means less efficient JOINs. The Uber guy is right that InnoDB handles this better as long as you don't touch the primary key (primary key updates in InnoDB are really bad). This is a common problem case we don't have an answer for yet. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why we lost Uber as a user
On 07/26/2016 09:54 AM, Joshua D. Drake wrote: > Hello, > > The following article is a very good look at some of our limitations and > highlights some of the pains many of us have been working "around" since > we started using the software. They also had other reasons to switch to MySQL, particularly around changes of staffing (the switch happened after they got a new CTO). And they encountered that 9.2 bug literally the week we released a fix, per one of the mailing lists. Even if they switched off, it's still a nice testimonial that they once ran their entire worldwide fleet off a single Postgres cluster. However, the issues they cite as limitations of our current replication system are real, or we wouldn't have so many people working on alternatives. We could really use pglogical in 10.0, as well as OLTP-friendly MM replication. The write amplification issue, and its correllary in VACUUM, certainly continues to plague some users, and doesn't have any easy solutions. I do find it interesting that they mention schema changes in passing, without actually saying anything about them -- given that schema changes have been one of MySQL's major limitations. I'll also note that they don't mention any of MySQL's corresponding weak spots, such as limitations on table size due to primary key sorting. One wonders what would have happened if they'd adopted a sharding model on top of Postgres? I would like to see someone blog about our testing for replication corruption issues now, in response to this. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 06/16/2016 11:01 PM, Craig Ringer wrote: > > I thought about raising this, but I think in the end it's replacing one > confusing and weird versioning scheme for another confusing and weird > versioning scheme. > > It does have the advantage that that compare a two-part major like > 090401 vs 090402 won't be confused when they compare 100100 and 100200, > since it'll be 11 and 12. So it's more backward-compatible. But > ugly. > Realistically, though, we're more likely to end up with 10.0.1 than 10.1. I don't think we're anywhere near plumbing the depths of the stuff which will break because folks are parsing our version numbers with regexes. In more major software, this will break nagios check_postgres. I'm not happy with it, but I believe that's where we'll end up. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 06/17/2016 10:04 AM, Alvaro Herrera wrote: > Merlin Moncure wrote: > >> Ugliness is a highly subjective qualifier. OTOH, Backwards >> compatibility, at least when the checks are properly written :-), is a >> very objective benefit. > > This is the argument that made us kept the PostgreSQL name instead of > renaming back to Postgres. I'm not a fan of it. > Well ... no. We kept the PostgreSQL name for three reasons. Back in 2005, which was the last time we could have reasonably changed it, nobody had the time/energy to do all of the search-and-replace-and-contact-every-packager required. The folks who were most enthusiastic about the change wanted someone else to do the work. Plus, our Japanese community, which was like 40% of our worldwide community at the time, was opposed to the change. The third reason is that we have a registered trademark on "PostgreSQL", but "postgres" is public domain. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 06/07/2016 11:01 PM, Robert Haas wrote:
> On Fri, Jun 3, 2016 at 9:39 AM, Tom Lane wrote:
>> Robert Haas writes:
>>> I think we should just go with max_parallel_workers for a limit on
>>> total parallel workers within max_work_processes, and
>>> max_parallel_workers_per_gather for a per-Gather limit. It's slightly
>>> annoying that we may end up renaming the latter GUC, but not as
>>> annoying as spending another three weeks debating this and missing
>>> beta2.
>>
>> +1. I'm not as convinced as you are that we'll replace the GUC later,
>> but in any case this is an accurate description of the current
>> semantics. And I'm really *not* in favor of fudging the name with
>> the intent of changing the GUC's semantics later --- that would fail
>> all sorts of compatibility expectations.
>
> Here's a patch change max_parallel_degree to
> max_parallel_workers_per_gather, and also changing parallel_degree to
> parallel_workers. I haven't tackled adding a separate
> max_parallel_workers, at least not yet. Are people OK with this?
+1
>
> Note that there is a dump/restore hazard if people have set the
> parallel_degree reloption on a beta1 install, or used ALTER { USER |
> DATABASE } .. SET parallel_degree. Can everybody live with that?
> Should I bump catversion when applying this?
IMHO, we just need to call it out in the beta2 announcement.
--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUGS] Routine analyze of single column prevents standard autoanalyze from running at all
On 06/06/2016 01:38 PM, Tom Lane wrote: > Also, I'd be a bit inclined to disable the counter reset whenever a column > list is specified, disregarding the corner case where a list is given but > it includes all the table's analyzable columns. It doesn't really seem > worth the effort to account for that case specially (especially after > you consider that index expressions should count as analyzable columns). > > Thoughts? +1. Better to err on the side of duplicate analyzes than none at all. Also, I'm not surprised this took so long to discover; I doubt most users are aware that you *can* analyze individual columns. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 06/02/2016 09:33 PM, Peter Eisentraut wrote: > On 6/3/16 12:21 AM, Petr Jelinek wrote: >> On 01/06/16 17:55, David G. Johnston wrote: >>> On Wed, Jun 1, 2016 at 11:45 AM, Petr Jelinek >> <mailto:p...@2ndquadrant.com>>wrote: >>> >>> That GUC also controls worker processes that are started by >>> extensions, not just ones that parallel query starts. This is btw >>> one thing I don't like at all about how the current limits work, the >>> parallel query will fight for workers with extensions because they >>> share the same limit. >>> >>> >>> Given that this models reality the GUC is doing its job. Now, maybe we >>> need additional knobs to give the end-user the ability to influence how >>> those fights will turn out. >> >> Agreed, my point is that I think we do need additional knob. > > We need one knob to control how many process slots to create at server > start, and then a bunch of sliders to control how to allocate those > between regular connections, superuser connections, replication, > autovacuum, parallel workers, background workers (by tag/label/group), > and so on. Now that's crazy talk. I mean, next thing you'll be saying that we need the ability to monitor this, or even change it at runtime. Where does the madness end? ;-) Seriously, you have a point here; it's maybe time to stop tackling process management per server piecemeal. Question is, who wants to work on this? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 06/02/2016 01:42 PM, David G. Johnston wrote: > Are you referring to right now or if we move the goal posts to making > > this a per-statement reservation? > > I was assuming that we would have *both* per-operation and per-statement > limits. I can see reasons for having both, I can see why power users > would want both, but it's going to be overwhelming to casual users. > > > Got that. The only problem on that front with the current setup is > that right now we are saying: "at most use 3 of the 8 available > processes": i.e., we tie ourselves to a fixed number when I think a > better algorithm would be: "on/off/auto - default auto" and we detect at > runtime whatever values we feel are most appropriate based upon the > machine we are running on. If the user doesn't like our choices they > can specify their own values. But the only specified values in the > configurations would be those placed there automatically by the user. > If value isn't specified but is required it gets set at startup and can > be seen in pg_settings. > Well, the hard part here is that the appropriate value is based on the level of concurrency in the database as a whole. For example, if it's a website database with 200 active connections on a 32-core machine already, you want zero parallelism. Whereas if it's the only current statement on a 16-core VM, you want like 8. That sounds like a heuristic, except that the number of concurrent statements can change in milleseconds. So we'd really want to base this on some sort of moving average, set conservatively. This will be some interesting code, and will probably need to be revised several times before we get it right. Particularly since this would involve scanning some of the global catalogs we've been trying to move activity off of. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 06/02/2016 01:08 PM, David G. Johnston wrote: > On Thu, Jun 2, 2016 at 3:52 PM, Josh berkus <mailto:j...@agliodbs.com>>wrote: > > On 06/02/2016 08:53 AM, Tom Lane wrote: > > Josh berkus mailto:j...@agliodbs.com>> writes: > >> On 06/02/2016 04:58 AM, Robert Haas wrote: > >>> Well, I think we could drop node, if you like. I think parallel > >>> wouldn't be good to drop, though, because it sounds like we want a > >>> global limit on parallel workers also, and that can't be just > >>> max_workers. So I think we should keep parallel in there for all of > >>> them, and have max_parallel_workers and > >>> max_parallel_workers_per_gather(_node). The reloption and the Path > >>> struct field can be parallel_workers rather than parallel_degree. > > > >> So does that mean we'll rename it if you manage to implement a > parameter > >> which controls the number of workers for the whole statement? > > > > That would fit in as something like max_parallel_workers_per_statement. > > ETOOMANYKNOBS > > I'm trying to think of some way we can reasonably automate this for > users ... > > > Are you referring to right now or if we move the goal posts to making > this a per-statement reservation? I was assuming that we would have *both* per-operation and per-statement limits. I can see reasons for having both, I can see why power users would want both, but it's going to be overwhelming to casual users. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 06/02/2016 08:53 AM, Tom Lane wrote: > Josh berkus writes: >> On 06/02/2016 04:58 AM, Robert Haas wrote: >>> Well, I think we could drop node, if you like. I think parallel >>> wouldn't be good to drop, though, because it sounds like we want a >>> global limit on parallel workers also, and that can't be just >>> max_workers. So I think we should keep parallel in there for all of >>> them, and have max_parallel_workers and >>> max_parallel_workers_per_gather(_node). The reloption and the Path >>> struct field can be parallel_workers rather than parallel_degree. > >> So does that mean we'll rename it if you manage to implement a parameter >> which controls the number of workers for the whole statement? > > That would fit in as something like max_parallel_workers_per_statement. ETOOMANYKNOBS I'm trying to think of some way we can reasonably automate this for users ... -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 06/02/2016 04:58 AM, Robert Haas wrote: > Well, I think we could drop node, if you like. I think parallel > wouldn't be good to drop, though, because it sounds like we want a > global limit on parallel workers also, and that can't be just > max_workers. So I think we should keep parallel in there for all of > them, and have max_parallel_workers and > max_parallel_workers_per_gather(_node). The reloption and the Path > struct field can be parallel_workers rather than parallel_degree. So does that mean we'll rename it if you manage to implement a parameter which controls the number of workers for the whole statement? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 06/01/2016 02:21 PM, Robert Haas wrote: > If you lined up ten people in a room all of whom were competent > database professionals and none of whom knew anything about PostgreSQL > and asked them to guess what a setting called work_mem does and what a > setting called max_parallel_degree does, I will wager you $5 that > they'd do better on the second one. Likewise, I bet the guesses for > max_parallel_degree would be closer to the mark than the guesses for > maintenance_work_mem or replacement_sort_tuples or commit_siblings or > bgwriter_lru_multiplier. Incidentally, the reason I didn't jump into this thread until the patches showed up is that I don't think it actually matters what the parameters are named. They're going to require documentation regardless, parallism just isn't something people grok instinctively. I care about how the parameters *work*, and whether that's consistent across our various resource management settings. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 01:04 PM, Tom Lane wrote: > The name should be closely related to what we use for #3. I could go for > max_total_parallel_workers for #2 and max_parallel_workers for #3. > Or maybe max_parallel_workers_total? How about parallel_worker_pool? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Logic behind parallel default? WAS: Rename max_parallel_degree?
On 05/31/2016 11:10 AM, Tom Lane wrote: > Josh berkus writes: >> Is there a thread on how we determined this default of 2? I can't find >> one under likely search terms. > > The 9.6 open-items list cites > > https://www.postgresql.org/message-id/flat/20160420174631.3qjjhpwsvvx5b...@alap3.anarazel.de Looks like we didn't decide for the release, just the beta. I can see two ways to go for the final release: 1. Ship with max_parallel_X = 2 (or similar) and a very low max_worker_processes (e.g. 4). 2. Ship with max_parallel_X = 1 (or 0, depending), and with a generous max_worker_processes (e.g. 16). Argument in favor of (1): we want parallelism to work out of the gate for users running on low-concurrency systems. These settings would let some parallelism happen immediately, without overwhelming a 4-to-8-core system/vm. Tuning for the user would then be fairly easy, as we could just tell them "set max_worker_processes to half the number of cores you have". Argument in favor of (2): parallelism is potentially risky for .0, and as a result we want it disabled unless users choose to enable it. Also, defaulting to off lets users make more use of the parallel_degree table attribute to just enable parallelism on select tables. Thoughts? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 11:29 AM, Tom Lane wrote: > Josh berkus writes: >> One more consistency question: what's the effect of running out of >> max_parallel_workers? > > ITYM max_worker_processes (ie, the cluster-wide pool size)? Yes. Sorry for contributing to the confusion. Too many similar-sounding parameter names. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 11:27 AM, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 11:22 AM, Josh berkus wrote: >>> I think we can hope that developers are going to be less confused about >>> that than users. >> >> Makes sense. > > Maybe EXPLAIN doesn't have to use the term parallel worker at all. It > can instead use a slightly broader terminology, possibly including the > term "core". > >> One more consistency question: what's the effect of running out of >> max_parallel_workers? >> >> That is, say max_parallel_workers is set to 10, and 8 are already >> allocated. If I ask for max_parallel_X = 4, how many cores to I use? > > Well, it depends on the planner, of course. But when constrained only > by the availability of worker processes, then your example could use 3 > cores -- the 2 remaining parallel workers, plus the leader itself. > >> Presumably the leader isn't counted towards max_parallel_workers? (oops, I meant max_worker_processes above) So, there's six things we'd like to make consistent to limit user confusion: 1. max_parallel_X and number of cores used 2. max_parallel_X and EXPLAIN output 3. max_parallel_X and gatekeeping via max_worker_processes 4. max_parallel_X and parallelism of operations (i.e. 2 == 2 parallel scanners) 5. settings in other similar databases (does someone have specific citations for this)? 6. consistency with other GUCs (0 or -1 to disable settings) We can't actually make all five things consistent, as some of them are contradictory; for example, (1) and (3) cannot be reconciled. So we need to evaluate which things are more likely to cause confusion. My general evaluation would be to make the GUC be the number of additional workers used, not the total number (including the leader). From my perspective, that makes (2), (3) and (6) consistent, and (4) cannot be made consistent because different types of parallel nodes behave different ways (i.e, some are parallel with only 1 additional worker and some are not). However, I'm resigned to the fact that user confusion is inevitable whichever way we choose, and could be persuaded the other way. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 11:17 AM, Peter Eisentraut wrote: > On 5/31/16 2:02 PM, Josh berkus wrote: >> I get where you're coming from, but I think Haas's query plan output is >> going to show us the confusion we're going to get. So we need to either >> change the parameter, the explain output, or brace ourselves for endless >> repeated questions. > > Changing the explain output doesn't sound so bad to me. > > The users' problem is that the parameter setting ought to match the > EXPLAIN output. > > The developers' problem is that the EXPLAIN output actually corresponds > to leader + (N-1) workers internally. > > I think we can hope that developers are going to be less confused about > that than users. Makes sense. One more consistency question: what's the effect of running out of max_parallel_workers? That is, say max_parallel_workers is set to 10, and 8 are already allocated. If I ask for max_parallel_X = 4, how many cores to I use? Presumably the leader isn't counted towards max_parallel_workers? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Logic behind parallel default? WAS: Rename max_parallel_degree?
On 05/31/2016 11:00 AM, Tom Lane wrote: > ! If this occurs, the plan will run with fewer workers than expected, > ! which may be inefficient. The default value is 2. Setting this > ! value to 0 disables parallel query execution. Is there a thread on how we determined this default of 2? I can't find one under likely search terms. I'm concerned about the effect of overallocating parallel workers on systems which are already running out of cores (e.g. AWS instances), and run with default settings. Possibly max_parallel_workers takes care of this, which is why I want to understand the logic here. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 10:51 AM, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 10:46 AM, Josh berkus wrote: >> In parallel seq scan and join, do the "masters" behave as workers as well? > > It depends. They will if they can. If the parallel seq scan leader > isn't getting enough work to do from workers (enough tuples to process > from the shared memory queue), it will start acting as a worker fairly > quickly. > With parallel aggregate, and some other cases, that will always happen. > > Even when the leader is consuming input from workers, that's still perhaps > pegging one CPU core. So, it doesn't really invalidate what I said about > the number of cores being the primary consideration. > I get where you're coming from, but I think Haas's query plan output is going to show us the confusion we're going to get. So we need to either change the parameter, the explain output, or brace ourselves for endless repeated questions. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 10:38 AM, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 10:23 AM, Josh berkus wrote: >> It's still WAY simpler to understand "max_parallel is the number of >> parallel workers I requested". > > (Sorry Josh, somehow hit reply, not reply-all) > > Yes, it is. But as long as parallel workers are not really that > distinct to the leader-as-worker when executing a parallel query, then > you have another consideration. Which is that you need to care about > how many cores your query uses first and foremost, and not the number > of parallel workers used. I don't think that having only one worker > will cause too much confusion, because users will trust that we won't > allow something that simply makes no sense to happen. > > In my parallel create index patch, the leader participates as a worker > to scan and sort runs. It's identical to a worker, practically > speaking, at least until time comes to merge those runs. Similarly, > parallel aggregate does not really have much for the leader process to > do other than act as a worker. In parallel seq scan and join, do the "masters" behave as workers as well? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 10:16 AM, Peter Geoghegan wrote: > On Tue, May 31, 2016 at 10:10 AM, Josh berkus wrote: >> "max_parallel_degree is the amount of parallelism in the query, with the >> understanding that the original parent process counts as 1, which means >> that if you set it to 1 you get no parallelism, and if you want 4 >> parallel workers you need to set it to 5." >> >> Which one of those is going to require more explanations on -general and >> -novice? Bets? >> >> Let's not be complicated for the sake of being complicated. > > But the distinction between parallel workers and backends that can > participate in parallel query does need to be user-visible. Worker > processes are a commodity (i.e. the user must consider > max_worker_processes). It's still WAY simpler to understand "max_parallel is the number of parallel workers I requested". Any system where you set it to 2 and get only 1 worker on an idle system is going to cause endless queries on the mailing lists. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 10:03 AM, Tom Lane wrote: > Josh berkus writes: >> I realize there's a lot of water under the bridge here, but I think >> we're going to get 1000 questions on -general of the type: "I asked for >> 8 parallel workers, why did I only get 7?". I believe we will regret >> this change. >> So, one vote from me to revert. > > Well, that gets back to the question of whether average users will > understand the "degree" terminology. For the record, while I do not > like the current behavior either, this was not the solution I favored. > I thought we should rename the GUC and keep it as meaning the number > of additional worker processes. I will happily bet anyone a nice dinner in Ottawa that most users will not understand it. Compare this: "max_parallel is the maximum number of parallel workers which will work on each stage of the query which is parallizable. If you set it to 4, you get up to 4 workers." with this: "max_parallel_degree is the amount of parallelism in the query, with the understanding that the original parent process counts as 1, which means that if you set it to 1 you get no parallelism, and if you want 4 parallel workers you need to set it to 5." Which one of those is going to require more explanations on -general and -novice? Bets? Let's not be complicated for the sake of being complicated. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Rename max_parallel_degree?
On 05/31/2016 09:15 AM, Robert Haas wrote: > On Sun, May 29, 2016 at 1:33 AM, Noah Misch wrote: >> On Fri, May 06, 2016 at 02:52:30PM -0400, Robert Haas wrote: >>> OK, my reading of this thread is that there is a consensus is to >>> redefine max_parallel_degree=1 as "no parallelism" and >>> max_parallel_degree>1 as "parallelism using a leader plus N-1 >>> workers", and along with that, to keep the names unchanged. However, >>> I don't think I can get that done before beta1, at least not without a >>> serious risk of breaking stuff. I can look at this post-beta1. >> >> [This is a generic notification.] >> >> The above-described topic is currently a PostgreSQL 9.6 open item. Robert, >> since you committed the patch believed to have created it, you own this open >> item. If some other commit is more relevant or if this does not belong as a >> 9.6 open item, please let us know. Otherwise, please observe the policy on >> open item ownership[1] and send a status update within 72 hours of this >> message. Include a date for your subsequent status update. Testers may >> discover new open items at any time, and I want to plan to get them all fixed >> well in advance of shipping 9.6rc1. Consequently, I will appreciate your >> efforts toward speedy resolution. Thanks. > > Here is a patch. Note that I still don't agree with this change, but > I'm bowing to the will of the group. > > I think that some of the people who were in favor of this change > should review this patch, including especially the language I wrote > for the documentation. If that happens, and the reviews are positive, > then I will commit this. If that does not happen, then I will > interpret that to mean that there isn't actually all that much > interest in changing this after all and will accordingly recommend > that this open item be removed without further action. > > Here is a test which shows how it works: > > rhaas=# set max_parallel_degree = 100; > SET > rhaas=# alter table pgbench_accounts set (parallel_degree = 10); > ALTER TABLE > rhaas=# explain (analyze) select count(*) from pgbench_accounts; > > QUERY PLAN > > Finalize Aggregate (cost=177436.04..177436.05 rows=1 width=8) > (actual time=383.244..383.244 rows=1 loops=1) >-> Gather (cost=177435.00..177436.01 rows=10 width=8) (actual > time=383.040..383.237 rows=9 loops=1) > Workers Planned: 9 > Workers Launched: 8 I realize there's a lot of water under the bridge here, but I think we're going to get 1000 questions on -general of the type: "I asked for 8 parallel workers, why did I only get 7?". I believe we will regret this change. So, one vote from me to revert. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding an alternate syntax for Phrase Search
On 05/22/2016 06:53 PM, Teodor Sigaev wrote:
>
>> to_tsquery(' Berkus & "PostgreSQL Version 10.0" ')
>>
>> ... would be equivalent to:
>>
>> to_tsquery(' Berkus & ( PostgreSQL <-> version <-> 10.0 )')
>
> select to_tsquery('Berkus') && phraseto_tsquery('PostgreSQL Version 10.0');
> does it as you wish
Aha, you didn't mention this in your presentation. That seems plenty
good enough for 9.6.
--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Adding an alternate syntax for Phrase Search
Folks,
This came up at pgCon.
The 'word <-> word <-> word' syntax for phrase search is not
developer-friendly. While we need the <-> operator for SQL and for the
sophisticated cases, it would be really good to support an alternate
syntax for the simplest case of "words next to each other". My proposal
is enclosing the phrase in double-quotes, which would be intuitive to
users and familiar from search engines. Thus:
to_tsquery(' Berkus & "PostgreSQL Version 10.0" ')
... would be equivalent to:
to_tsquery(' Berkus & ( PostgreSQL <-> version <-> 10.0 )')
I realize we're already in beta, but pgCon was actually the first time I
saw the new syntax. I think if we don't do this now, we'll be doing it
for 10.0.
--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Reviewing freeze map code
On 05/18/2016 03:51 PM, Peter Geoghegan wrote: > On Wed, May 18, 2016 at 8:52 AM, Jeff Janes wrote: >> How about going with something that says more about why we are doing >> it, rather than trying to describe in one or two words what it is >> doing? >> >> VACUUM (FORENSIC) >> >> VACUUM (DEBUG) >> >> VACUUM (LINT) > > +1 Maybe this is the wrong perspective. I mean, is there a reason we even need this option, other than a lack of any other way to do a full table scan to check for corruption, etc.? If we're only doing this for integrity checking, then maybe it's better if it becomes a function, which could be later extended with additional forensic features? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 05/13/2016 07:18 PM, Mark Dilger wrote: > My main concern is that a commitment to never, ever break backwards > compatibility is a commitment to obsolescence. It therefore makes sense to > reserve room in the numbering scheme to be clear and honest about when > backwards compatibility has been broken. The major number is the normal > place to do that. The problem with that idea is that *minor* backwards compatibility breakage is much more likely in each-and-every version than major breakage is at any time in the foreseeable future. The last major breakage we really had was version 8.3, which if we'd been going by compatibility should have been 9.0 (also for other reasons). And if we adopt the "backwards compatibility" approach, then we'll just be switching from the argument we're having now to the argument of "is this enough breakage to rate a .0? Yes/No?". Which argument will be just as long as this one. So, my vote is now +1 to go to the 2-part numbering scheme. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 05/13/2016 05:22 PM, Mark Dilger wrote: >>> >> Any project that starts inflating its numbering scheme sends a message to >>> >> users of the form, "hey, we've just been taken over by marketing people, >>> >> and >>> >> software quality will go down from now on." >> > >> > I don't think this is about version number inflation, but actually more >> > the opposite. What you're calling the major number is really a marketing >> > number. There is not a technical distinction between major releases where >> > we choose to bump the first number and those where we choose to bump the >> > second. It's all about marketing. So to me, merging those numbers would >> > be an anti-marketing move. I think it's a good move: it would be more >> > honest and transparent about what the numbers mean, not less so. > I find your argument persuasive if there is no possibility of ever needing > a major number to bump. But if anything like what I described above could > someday happen, it seems the major.minor.micro format would come in > handy. Perhaps the problem (from my perspective) is that the major number > has been used for purely marketing purposes in the past, and I've tried to > avert my eyes to that. But going forward, my vote (worth less than half a > cent I'm sure) is to stop using it for marketing reasons. Per a long discussion on -advocacy, nobody has any specific plans to do substantial breakage of backwards compatibility. Theoretically we might someday want to change the on-disk format, but nobody has plans to do so in the immediate future. How long should we hold out for that? Until 9.27? And I don't find dropping the "money" type to be substantial breakage. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 05/13/2016 02:00 PM, Tom Lane wrote: > I still don't like that much, and just thought of another reason why: > it would foreclose doing two major releases per year. We have debated > that sort of schedule in the past. While I don't see any reason to > think we'd try to do it in the near future, it would be sad if we > foreclosed the possibility by a poor choice of versioning scheme. Well, we have done two major releases in a year before, mostly due to one release being late and the succeeding one being on time. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0
On 05/13/2016 01:04 PM, Joshua D. Drake wrote: > On 05/13/2016 12:03 PM, Josh berkus wrote: >> On 05/13/2016 11:48 AM, Robert Haas wrote: >>> On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake >>> wrote: > >> Anyway, all of this is a moot point, because nobody has the power to >> tell the various companies what to do. We're just lucky that everyone >> is still committed to writing stuff which adds to PostgreSQL. > > Lucky? No. We earned it. We earned it through years and years of hard > work. Should we be thankful? Absolutely. Should we be grateful that we > have such a powerful and engaged commercial contribution base? 100%. Lucky. Sure there was work and personal integrity involved, but like any success story, there was luck. But we've also been fortunate in not spawning hostile-but-popular forks by people who left the project, and that none of the companies who created hostile forks were very successful with them, and that nobody has seriously tried using lawyers to control/ruin the project. And, most importantly, we've been lucky that a lot of competing projects have self-immolated instead of being successful and brain-draining our contributors (MySQL, ANTS, MonetDB, etc.) -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 05/13/2016 01:15 PM, Robert Haas wrote: > On Fri, May 13, 2016 at 2:49 PM, Tom Lane wrote: >> So I think we should solve these problems at a stroke, and save ourselves >> lots of breath in the future, by getting rid of the whole "major major" >> idea and going over to a two-part version numbering scheme. To be >> specific: >> >> * This year's major release will be 9.6.0, with minor updates 9.6.1, >> 9.6.2, etc. It's too late to do otherwise for this release cycle. >> >> * Next year's major release will be 10.0, with minor updates 10.1, >> 10.2, etc. >> >> * The year after, 11.0. Etc cetera. >> >> No confusion, no surprises, no debate ever again about what the next >> version number is. >> >> This is by no means a new idea, but I think its time has come. > > Man, I hate version number inflation. I'm running Firefox 45.0.2, and > I think that's crazy. It hit 1.0 when were at aversion 7.4! Granted, > this wouldn't be that bad, but I have always thought that burning > through a first digit a few times a decade is much more sensible than > doing it every year. We just have to remember to bump the first digit > occasionally. Well, FF has this issue because they release a new version every 6 weeks. Even bumping once per year, we wouldn't hit version 20 until 2027. > If we don't want to stick with the current practice of debating when > to bump the same digit, then let's agree that 10.0 will follow 9.6 and > after that we'll bump the first digit after X.4, as we did with 7.X > and 8.X. Why X.4? Seems arbitrary. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 05/13/2016 11:49 AM, Tom Lane wrote: > Alvaro Herrera writes: >> Josh berkus wrote: >>> Anyway, can we come up with a consensus of some minimum changes it will >>> take to make the next version 10.0? > >> I think the next version should be 10.0 no matter what changes we put >> in. > > Here's my two cents: we called 8.0 that on the basis of the native Windows > port, and 9.0 that on the basis of getting in-core replication support. > Both of those were game-changing features that deserved a "major major" > version number bump. But as the project matures, it gets harder and > harder to come up with game-changing features in the span of a single > release. Parallel query is a great example: a fully mature parallel query > feature would, IMO, easily justify a 10.0 moniker. But what we have today > is more like half or two-thirds of a feature. If we call this release > 10.0 on the strength of that, we could justifiably be accused of > overselling it. On the other hand, if we wait till next year when > parallelism presumably will be much more fully baked, it'll be a bit > anticlimactic to call it 10.0 then. This isn't going to get better with > other major features that can be expected to appear in future. So we can > expect to continue to waste lots of time debating the "what to call it" > question, in pretty much every year except for the one or two immediately > after a "major major" bump. > > There's also the problem that the current numbering scheme confuses > people who aren't familiar with the project. How many times have > you seen people say "I'm using Postgres 8" or "Postgres 9" when asked > what version they're on? > > So I think we should solve these problems at a stroke, and save ourselves > lots of breath in the future, by getting rid of the whole "major major" > idea and going over to a two-part version numbering scheme. To be > specific: > > * This year's major release will be 9.6.0, with minor updates 9.6.1, > 9.6.2, etc. It's too late to do otherwise for this release cycle. > > * Next year's major release will be 10.0, with minor updates 10.1, > 10.2, etc. > > * The year after, 11.0. Etc cetera. > > No confusion, no surprises, no debate ever again about what the next > version number is. > > This is by no means a new idea, but I think its time has come. I'm for it. Note that we will need to do a *bunch* of education around the change in version numbering schemes. And a bunch of people and packagers will need to change their version comparison scripts (while everyone should be using the sortable version numbers, not everyone does). So if we're going to make that change, I suggest doing it *now* to get the word out. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0
On 05/13/2016 11:48 AM, Robert Haas wrote: > On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake > wrote: >> Singular point contribution is not the point of my argument. My point is >> that if three people from EDB and three people from Citus got together and >> worked on a project in full collaboration it would be more beneficial to the >> project. > > Well, the scalability work in 9.6 went almost exactly like this, > assuming you count Andres as three people (which is entirely > reasonable) and Dilip, Mithun, Amit, and myself as three people (which > is maybe less reasonable, since I don't really want any of us counted > as less than a whole person). Frankly, PostgreSQL is practically a wonderland of inter-company collaboration. Yeah, there's some "does not play nice with others" which happens, but that's pretty much inevitable. Plus, it's also useful to have some companies go in different directions sometimes; the best approach to certain problems isn't always well defined. We might have a little more of that than is completely ideal, but it's rather hard to determine that. Anyway, all of this is a moot point, because nobody has the power to tell the various companies what to do. We're just lucky that everyone is still committed to writing stuff which adds to PostgreSQL. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 05/13/2016 08:19 AM, Bruce Momjian wrote: >> > Thoughts? Is it crazy to go from 9.6beta1 to 10.0beta2? What would >> > actually be involved in making the change? > Someone mentioned how Postgres 8.5 became 9.0, but then someone else > said the change was made during alpha releases, not beta. Can someone > dig up the details? /me digs through the announcement archives ... We changed it to 9.0 for Alpha4. By Beta1, it was already 9.0. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 10.0
On 05/13/2016 11:31 AM, Alvaro Herrera wrote: > Josh berkus wrote: > >> Anyway, can we come up with a consensus of some minimum changes it will >> take to make the next version 10.0? > > I think the next version should be 10.0 no matter what changes we put > in. > Well, if we adopt 2-part version numbers, it will be. Maybe that's the easiest thing? Then we never have to have this discussion again, which certainly appeals to me ... -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
