[HACKERS] less log level for success dynamic background workers for 9.5

[Pavel Stehule]

Hi

I am working on scheduler extension for 9.5. It use bgworkers intensively
for any task. This is reason, why I need to decrease a log level - and I am
thinking so parallel computing needs it due high number of created and
finished workers.

It should be fixed in 9.5 - because it is limiting factor for bgworkers
usage in 9.5.

Regards

Pavel
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
new file mode 100644
index f57224c..68c9505
*** a/src/backend/postmaster/bgworker.c
--- b/src/backend/postmaster/bgworker.c
*** BackgroundWorkerStateChange(void)
*** 341,347 
  		rw->rw_terminate = false;
  
  		/* Log it! */
! 		ereport(LOG,
  (errmsg("registering background worker \"%s\"",
  		rw->rw_worker.bgw_name)));
  
--- 341,347 
  		rw->rw_terminate = false;
  
  		/* Log it! */
! 		ereport(DEBUG1,
  (errmsg("registering background worker \"%s\"",
  		rw->rw_worker.bgw_name)));
  
*** ForgetBackgroundWorker(slist_mutable_ite
*** 370,376 
  	slot = &BackgroundWorkerData->slot[rw->rw_shmem_slot];
  	slot->in_use = false;
  
! 	ereport(LOG,
  			(errmsg("unregistering background worker \"%s\"",
  	rw->rw_worker.bgw_name)));
  
--- 370,376 
  	slot = &BackgroundWorkerData->slot[rw->rw_shmem_slot];
  	slot->in_use = false;
  
! 	ereport(DEBUG1,
  			(errmsg("unregistering background worker \"%s\"",
  	rw->rw_worker.bgw_name)));
  
*** RegisterBackgroundWorker(BackgroundWorke
*** 741,747 
  	static int	numworkers = 0;
  
  	if (!IsUnderPostmaster)
! 		ereport(LOG,
  		 (errmsg("registering background worker \"%s\"", worker->bgw_name)));
  
  	if (!process_shared_preload_libraries_in_progress)
--- 741,747 
  	static int	numworkers = 0;
  
  	if (!IsUnderPostmaster)
! 		ereport(DEBUG1,
  		 (errmsg("registering background worker \"%s\"", worker->bgw_name)));
  
  	if (!process_shared_preload_libraries_in_progress)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
new file mode 100644
index 324bf7a..633cdd7
*** a/src/backend/postmaster/postmaster.c
--- b/src/backend/postmaster/postmaster.c
*** CleanupBackgroundWorker(int pid,
*** 2964,2970 
  		rw->rw_child_slot = 0;
  		ReportBackgroundWorkerPID(rw);	/* report child death */
  
! 		LogChildExit(LOG, namebuf, pid, exitstatus);
  
  		return true;
  	}
--- 2964,2971 
  		rw->rw_child_slot = 0;
  		ReportBackgroundWorkerPID(rw);	/* report child death */
  
! 		LogChildExit(EXIT_STATUS_0(exitstatus) ? DEBUG1 :  LOG,
! 			namebuf, pid, exitstatus);
  
  		return true;
  	}
*** do_start_bgworker(RegisteredBgWorker *rw
*** 5420,5426 
  {
  	pid_t		worker_pid;
  
! 	ereport(LOG,
  			(errmsg("starting background worker process \"%s\"",
  	rw->rw_worker.bgw_name)));
  
--- 5421,5427 
  {
  	pid_t		worker_pid;
  
! 	ereport(DEBUG1,
  			(errmsg("starting background worker process \"%s\"",
  	rw->rw_worker.bgw_name)));
  

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Michael Paquier]

On Sun, Jun 14, 2015 at 12:30 PM, Bruce Momjian wrote:
> On Sun, Jun 14, 2015 at 11:44:25AM +0900, Tatsuo Ishii wrote:
>> The real question here is, if contributors are required to show their
>> "official" names (along with their alias or nickname if any).  If yes,
>> it is apparent that all contributor names should be shown in a unified
>> formal style.
>>
>> If not, we could leave as it is.
>>
>> BTW, for Japanese almost 99% of Japanese names are obvious in that
>> which is the first name and which is the last name. So we don't need
>> to ask each contributor.
>
> OK, good.
>
>> > The names we currently have in the 9.5 release notes are:
>> >
>> > Fujii Masao
>> > Kyotaro Horiguchi
>> > Sawada Masahiko
>> > KaiGai Kohei
>> > Shigeru Hanada
>> > Etsuro Fujita
>> > Furuya Osamu
>> > MauMau
>> > Mitsumasa Kondo
>>
>> F.Y.I. here is a list rewritten in first-name/last-name style:
>>
>> Masao Fujii

Last name: Fujii, first name: Masao.

>> Kyotaro Horiguchi

Last name: Horiguchi, first name: Kyotaro

>> Masahiko Sawada

Last name; Sawada, first name: Masahiko.

>> Kohei KaiGai

Last name: Kaigai, first name Kohei

>> Shigeru Hanada

Last name: Hanada, first name: Shigeru

>> Etsuro Fujita

Last name: Fujita, first name: Esturo.

>> Osamu Furuya

Last name: Furuya, first name: Osamu.

>> Mitsumasa Kondo

Last name: Kondo, first name: Mitsumasa

>> MauMau

Last name: Mau, First name: Mau (??) or
Last name: None, first name: MauMau or
Last name: Maumau, first name: None.
In any case that's an interesting nickname.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Bruce Momjian]

On Sun, Jun 14, 2015 at 11:44:25AM +0900, Tatsuo Ishii wrote:
> The real question here is, if contributors are required to show their
> "official" names (along with their alias or nickname if any).  If yes,
> it is apparent that all contributor names should be shown in a unified
> formal style.
> 
> If not, we could leave as it is.
> 
> BTW, for Japanese almost 99% of Japanese names are obvious in that
> which is the first name and which is the last name. So we don't need
> to ask each contributor.

OK, good.

> > The names we currently have in the 9.5 release notes are:
> > 
> > Fujii Masao
> > Kyotaro Horiguchi
> > Sawada Masahiko
> > KaiGai Kohei
> > Shigeru Hanada
> > Etsuro Fujita
> > Furuya Osamu
> > MauMau
> > Mitsumasa Kondo
> 
> F.Y.I. here is a list rewritten in first-name/last-name style:
> 
> Masao Fujii

I have to admit I thought Fujii was his last name, then I thought it was
his first name, now you are saying it is his last name.  Obviously, I
am getting confused, and it does affect how I address him when I meet
him.

> Kyotaro Horiguchi
> Masahiko Sawada 
> Kohei KaiGai 
> Shigeru Hanada
> Etsuro Fujita
> Osamu Furuya 
> Mitsumasa Kondo
> MauMau
> 
> Note that I guess "MauMau" is a nickname.

I think we are fine consistenly putting Japanese last names first _if_
we always capitalize the last name, e.g. "FUJITA Etsuro" --- is that a
good idea? Does everyone like that?  Does any other country want that?

> > BTW, it is pretty cool to have contributor names that are natively in
> > scripts other than latin.  It is a pity that because of toolchain
> > limitations we cannot display names in kanji, cyrillic or other
> > character sets (in addition to their transliteration to latin script).
> 
> Japan User's Group (JPUG) recently starts to translate PostgreSQL doc
> using UTF-8 encoding. So technically this is possible. I'm not sure if
> developers want to have official document in other than ASCII though.

Ah, yes, you could do them special for that translation.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Problem with CREATE TABLE ... (LIKE ... INCLUDING INDEXES)

[Michael Paquier]

On Sun, Jun 14, 2015 at 11:38 AM, Thom Brown  wrote:
> As you can see, 3 indexes are missing, which happen to be ones that
> would duplicate the column definition of another index.  Is this
> intentional?  If so, shouldn't it be documented behaviour?

Looking at the code (transformIndexConstraints in parse_utilcmd.c),
this is intentional behavior:
/*
 * Scan the index list and remove any redundant index
specifications. This
 * can happen if, for instance, the user writes UNIQUE PRIMARY KEY. A
 * strict reading of SQL would suggest raising an error
instead, but that
 * strikes me as too anal-retentive. - tgl 2001-02-14
 *
 * XXX in ALTER TABLE case, it'd be nice to look for duplicate
 * pre-existing indexes, too.
 */
Per this commit:
commit: c7d2ce7bc6eb02eac0c10fae9caf2936a71ad25c
author: Tom Lane 
date: Wed, 14 Feb 2001 23:32:38 +
Repair problems with duplicate index names generated when CREATE TABLE
specifies redundant UNIQUE conditions.

Perhaps a mention in the docs in the page of CREATE TABLE would be
welcome. Something like "Redundant index definitions are ignored with
INCLUDING INDEXES".

Thoughts?
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Tatsuo Ishii]

> Bruce Momjian wrote:
>> I have committed the first draft of the 9.5 release notes.  You can view
>> the output here:
>> 
>>  http://momjian.us/pgsql_docs/release-9-5.html
> 
> I noticed something while reading this and would like the input of our
> Japanese contributors.
> 
> Normally, western names are written using the given name first, then the
> surname ("last name").  I am not well-versed on Japanese culture but my
> understanding is that they use their family name first, then their given
> name -- for instance, in "Yamada Tarō", Yamada is the family name and
> Tarō is the given name.
> 
> Apparently, when interacting with Western people, some Japanese seem to
> invert this and put the given name first.  Also, some write the family
> name in all caps -- I guess this is done so that it is clear which part
> is which.
> 
> Now, I think we should consider using a single style for all the
> Japanese names used in the release notes.  Can we have our Japanese
> contributors all agree on which style to use, and then let us know
> what's their name in that style?

The real question here is, if contributors are required to show their
"official" names (along with their alias or nickname if any).  If yes,
it is apparent that all contributor names should be shown in a unified
formal style.

If not, we could leave as it is.

BTW, for Japanese almost 99% of Japanese names are obvious in that
which is the first name and which is the last name. So we don't need
to ask each contributor.

> The names we currently have in the 9.5 release notes are:
> 
> Fujii Masao
> Kyotaro Horiguchi
> Sawada Masahiko
> KaiGai Kohei
> Shigeru Hanada
> Etsuro Fujita
> Furuya Osamu
> MauMau
> Mitsumasa Kondo

F.Y.I. here is a list rewritten in first-name/last-name style:

Masao Fujii
Kyotaro Horiguchi
Masahiko Sawada 
Kohei KaiGai 
Shigeru Hanada
Etsuro Fujita
Osamu Furuya 
Mitsumasa Kondo
MauMau

Note that I guess "MauMau" is a nickname.

> BTW, it is pretty cool to have contributor names that are natively in
> scripts other than latin.  It is a pity that because of toolchain
> limitations we cannot display names in kanji, cyrillic or other
> character sets (in addition to their transliteration to latin script).

Japan User's Group (JPUG) recently starts to translate PostgreSQL doc
using UTF-8 encoding. So technically this is possible. I'm not sure if
developers want to have official document in other than ASCII though.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Problem with CREATE TABLE ... (LIKE ... INCLUDING INDEXES)

[Thom Brown]

Hi all,

I've noticed that LIKE tablename INCLUDING INDEXES skips any indexes
that were duplicated.

e.g.

CREATE TABLE people (id int, alias text);
CREATE INDEX idx_people_id_1 ON people (id);
CREATE INDEX idx_people_id_2 ON people (id) WHERE id % 2 = 0;
CREATE INDEX idx_people_alias_1 ON people (alias);
CREATE INDEX idx_people_alias_2 ON people (alias);
CREATE INDEX idx_people_alias_3_tblspc ON people (alias) TABLESPACE ts;
CREATE INDEX idx_people_alias_4 ON people (alias) WITH (FILLFACTOR = 24);

\d+ people

Table "public.people"
 Column |  Type   | Modifiers | Storage  | Stats target | Description
+-+---+--+--+-
 id | integer |   | plain|  |
 alias  | text|   | extended |  |
Indexes:
"idx_people_alias_1" btree (alias)
"idx_people_alias_2" btree (alias)
"idx_people_alias_3_tblspc" btree (alias), tablespace "ts"
"idx_people_alias_4" btree (alias) WITH (fillfactor=24)
"idx_people_id_1" btree (id)
"idx_people_id_2" btree (id) WHERE (id % 2) = 0




CREATE SCHEMA test;
CREATE TABLE test.people (LIKE people INCLUDING INDEXES);

\d+ test.people

 Table "test.people"
 Column |  Type   | Modifiers | Storage  | Stats target | Description
+-+---+--+--+-
 id | integer |   | plain|  |
 alias  | text|   | extended |  |
Indexes:
"people_alias_idx" btree (alias)
"people_id_idx" btree (id)
"people_id_idx1" btree (id) WHERE (id % 2) = 0


As you can see, 3 indexes are missing, which happen to be ones that
would duplicate the column definition of another index.  Is this
intentional?  If so, shouldn't it be documented behaviour?

-- 
Thom


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Jan de Visser]

On June 13, 2015 09:30:12 PM Bruce Momjian wrote:
> Are there other countries where this would be appropriate?

I'm pretty sure Hungarian uses the family name-given name ordering, and it's 
also sometimes used in French, and therefore you often see French family names 
spelled in all caps. 



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Kouhei Kaigai]

> On Sat, Jun 13, 2015 at 08:18:26PM -0300, Alvaro Herrera wrote:
> > Bruce Momjian wrote:
> > > I have committed the first draft of the 9.5 release notes.  You can view
> > > the output here:
> > >
> > >   http://momjian.us/pgsql_docs/release-9-5.html
> >
> > I noticed something while reading this and would like the input of our
> > Japanese contributors.
> >
> > Normally, western names are written using the given name first, then the
> > surname ("last name").  I am not well-versed on Japanese culture but my
> > understanding is that they use their family name first, then their given
> > name -- for instance, in "Yamada Tarō", Yamada is the family name and
> > Tarō is the given name.
> >
> > Apparently, when interacting with Western people, some Japanese seem to
> > invert this and put the given name first.  Also, some write the family
> > name in all caps -- I guess this is done so that it is clear which part
> > is which.
> 
> Yes, I have Western-ordered all of them as best I could.
> 
> > Now, I think we should consider using a single style for all the
> > Japanese names used in the release notes.  Can we have our Japanese
> > contributors all agree on which style to use, and then let us know
> > what's their name in that style?
> >
> > The names we currently have in the 9.5 release notes are:
> >
> > Fujii Masao
> > Kyotaro Horiguchi
> > Sawada Masahiko
> > KaiGai Kohei
> > Shigeru Hanada
> > Etsuro Fujita
> > Furuya Osamu
> > MauMau
> > Mitsumasa Kondo
> 
> Yes, it would be good to know this.  The only problem is that I would
> have to _guess_ who is Japanese to do this consistently as the supplied
> names have varied orderings.  Do we really want to go there?  Are there
> other countries where this would be appropriate?
>
> > BTW, it is pretty cool to have contributor names that are natively in
> > scripts other than latin.  It is a pity that because of toolchain
> > limitations we cannot display names in kanji, cyrillic or other
> > character sets (in addition to their transliteration to latin script).
> 
> Yes, it would be cool to have the non-Latin name lettering for all
> submitters, then a Latin-ized version.  (I guess having my name in
> Armenian letters would be overkill ... hmmm.)  Anyway, Alvaro is right
> that our tooling doesn't have the capabilities to do this.  We could
> link to a webpage that did list all contributors with their native
> character-set names though.
>
I usually use the Eastern-style. KaiGai Kohei =  .
However, other people use their own manner.
Probably, it is not obvious for foreign people, and me also.
So, I don't think community need to take care.
How about a simply policy to show the name when contributors submitted
and discussed? If people prefer the Latin-style, they put their first
name first. Elsewhere, they put their family name first.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Alvaro Herrera]

Bruce Momjian wrote:
> On Sat, Jun 13, 2015 at 08:20:48PM -0300, Alvaro Herrera wrote:
> > Bruce Momjian wrote:
> > > I have committed the first draft of the 9.5 release notes.  You can view
> > > the output here:
> > > 
> > >   http://momjian.us/pgsql_docs/release-9-5.html
> > 
> > One more thing on names.  You have Alexander Shulgin as Alex Shulgin in
> > the second place his name appears.  Also, since he moved to Zalando (I
> 
> OK, good catch.  Fixed as "Alexander".
> 
> > guess to Berlin) he writes his name as Oleksandr.  I guess he'd have to
> > tell us which forms he would like to be used in the release notes.
> 
> Yes.  Does anyone have a current email address for him?  I assume his
> Command Prompt email address doesn't work, but I will try it here.

Oleksandr Shulgin 

See "PS" note near the bottom of
https://www.postgresql.org/message-id/CACACo5TjBAHYRJJc-U788DnfXbbPasX_nGUHFd%2B%3DcehA6SRs-w%40mail.gmail.com

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Bruce Momjian]

On Sat, Jun 13, 2015 at 08:18:26PM -0300, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> > 
> > http://momjian.us/pgsql_docs/release-9-5.html
> 
> I noticed something while reading this and would like the input of our
> Japanese contributors.
> 
> Normally, western names are written using the given name first, then the
> surname ("last name").  I am not well-versed on Japanese culture but my
> understanding is that they use their family name first, then their given
> name -- for instance, in "Yamada Tarō", Yamada is the family name and
> Tarō is the given name.
> 
> Apparently, when interacting with Western people, some Japanese seem to
> invert this and put the given name first.  Also, some write the family
> name in all caps -- I guess this is done so that it is clear which part
> is which.

Yes, I have Western-ordered all of them as best I could.

> Now, I think we should consider using a single style for all the
> Japanese names used in the release notes.  Can we have our Japanese
> contributors all agree on which style to use, and then let us know
> what's their name in that style?
> 
> The names we currently have in the 9.5 release notes are:
> 
> Fujii Masao
> Kyotaro Horiguchi
> Sawada Masahiko
> KaiGai Kohei
> Shigeru Hanada
> Etsuro Fujita
> Furuya Osamu
> MauMau
> Mitsumasa Kondo

Yes, it would be good to know this.  The only problem is that I would
have to _guess_ who is Japanese to do this consistently as the supplied
names have varied orderings.  Do we really want to go there?  Are there
other countries where this would be appropriate?

> BTW, it is pretty cool to have contributor names that are natively in
> scripts other than latin.  It is a pity that because of toolchain
> limitations we cannot display names in kanji, cyrillic or other
> character sets (in addition to their transliteration to latin script).

Yes, it would be cool to have the non-Latin name lettering for all
submitters, then a Latin-ized version.  (I guess having my name in
Armenian letters would be overkill ... hmmm.)  Anyway, Alvaro is right
that our tooling doesn't have the capabilities to do this.  We could
link to a webpage that did list all contributors with their native
character-set names though.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Bruce Momjian]

On Sat, Jun 13, 2015 at 03:53:43PM -0700, Peter Geoghegan wrote:
> On Thu, Jun 11, 2015 at 1:23 PM, Peter Geoghegan  wrote:
> > I'm not sure if it's worth mentioning the "cheap equality for text"
> > commit (e246b3d6eac09). I guess that it probably is, because it will
> > help with things like index scans, too. Arguably that isn't a sorting
> > thing (it's certainly not *just* a sorting thing).
> 
> I think we should really address this. Attached patch adds a new
> release note item for it. It also adds to the documentation that
> explains why users should prefer varchar(n)/text to character(n); the
> lack of abbreviated key support now becomes a huge disadvantage for
> character(n), whereas in previous versions the disadvantages were
> fairly minor.
> 
> In passing, I updated the existing sort item to reflect that only
> varchar(n), text, and numeric benefit from the abbreviation
> optimization (not character types more generally + numeric), and added
> a note on the effectiveness of the abbreviation optimization alone.

This all also seems like overkill to me.  Users just don't care about
this level of detail, and are easily confused by it.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Bruce Momjian]

On Sat, Jun 13, 2015 at 08:20:48PM -0300, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> > 
> > http://momjian.us/pgsql_docs/release-9-5.html
> 
> One more thing on names.  You have Alexander Shulgin as Alex Shulgin in
> the second place his name appears.  Also, since he moved to Zalando (I

OK, good catch.  Fixed as "Alexander".

> guess to Berlin) he writes his name as Oleksandr.  I guess he'd have to
> tell us which forms he would like to be used in the release notes.

Yes.  Does anyone have a current email address for him?  I assume his
Command Prompt email address doesn't work, but I will try it here.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Bruce Momjian]

On Sat, Jun 13, 2015 at 07:52:22AM -0300, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> > 
> > http://momjian.us/pgsql_docs/release-9-5.html
> > 
> > and it will eventually appear here:
> > 
> > http://www.postgresql.org/docs/devel/static/release.html
> > 
> > I am ready to make suggested adjustments, though I am traveling for
> > conferences for the next ten days so there might a delay in my replies.
> 
> Many people have commented on the "RETURN WHERE" text.  The item in
> question is this:
>   Have CREATE TABLE AS and REFRESH return an OID
> 
> This code was later changed to return ObjectAddress rather than OID;
> this is used to feed event triggers.  To answer the "where" question:
> the OID is returned to ProcessUtilitySlow, where it can be used by the
> DDL-command-collection infrastructure.
> 
> I don't think this particular change needs a mention in the release
> notes, since the ddl command collection is mentioned separately.  I
> would just remove it.

OK, removed.

> In the pg_upgrade section, I think this is a bit strangely worded:
>   Remove pg_upgrade support for 8.3 old clusters (Bruce Momjian)
> How about this?
>   Remove support for upgrading from 8.3 clusters (Bruce Momjian)
> 
> Why do we need a separate mention that pg_upgrade_support was moved to
> backend in the "Additional Modules" section?  Seems to me that having
> it mentioned in the pg_upgrade section would be better.

OK, moved.

> This entry is missing a word at the end:
>   Allow CURRENT/SESSION_USER to specify the current user in
>   certain

OK, wording updated, applied patch attached.

Thanks for the review.
-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +
diff --git a/doc/src/sgml/release-9.5.sgml b/doc/src/sgml/release-9.5.sgml
new file mode 100644
index ec49a63..478fd86
*** a/doc/src/sgml/release-9.5.sgml
--- b/doc/src/sgml/release-9.5.sgml
***
*** 802,808 

 
  Allow CURRENT/SESSION_USER to specify the
! current user in certain (Kyotaro Horiguchi, Álvaro Herrera)
 
  
 
--- 802,809 

 
  Allow CURRENT/SESSION_USER to specify the
! current user in some commands (Kyotaro Horiguchi, Álvaro
! Herrera)
 
  
 
***
*** 870,888 
 

  
-   
-
- Have CREATE
- TABLE AS and REFRESH
- return an OID (Álvaro Herrera)
-
- 
-
- WHERE IS THE OID RETURNED?
-
-   
- 
   
  
   
--- 871,876 
***
*** 1728,1738 
  

 
! Remove pg_upgrade support for 8.3 old clusters
! (Bruce Momjian)
 

  
   
  
  
--- 1716,1732 
  

 
! Remove support for upgrading from 8.3 clusters (Bruce Momjian)
 

  
+  
+   
+Move pg_upgrade_support code into backend and
+remove the module (Peter Eisentraut)
+   
+  
+ 
   
  
  
***
*** 2021,2033 

   
  
-  
-   
-Move pg_upgrade_support code into backend and
-remove the module (Peter Eisentraut)
-   
-  
- 
   

 Move dummy_seclabel, test_shm_mq,
--- 2015,2020 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Bruce Momjian]

On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote:
> Petr Jelinek wrote:
> > Hi,
> > 
> > +  
> > +   
> > +Add JSONB functions jsonb_set() and
> > +jsonb_pretty (Dmitry Dolgov, Andrew Dunstan)
> > +   
> > +  
> > 
> > I believe I should be 3rd author for this one as I rewrote large parts of
> > this functionality as part of the review.
> 
> Also, it looks like we could spell your last name with an accent,
> assuming the i-acute character in Latin1 is fine.

Ah, you are correct.  I found a few commits that did have that accent. 
All _seven_ of Petr's 9.5 commit mentions updated to add the accent. 
:-)  Thanks.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Bruce Momjian]

On Sat, Jun 13, 2015 at 08:25:55AM +0530, Amit Kapila wrote:
> On Sat, Jun 13, 2015 at 7:47 AM, Bruce Momjian  wrote:
> >
> > On Thu, Jun 11, 2015 at 09:02:35PM +0200, Magnus Hagander wrote:
> > > On Thu, Jun 11, 2015 at 8:56 PM, Josh Berkus  wrote:
> > >
> > >     On 06/10/2015 09:50 PM, Amit Kapila wrote:
> > >     > Also shall we mention about below in Migrations to 9.5 section
> > >     >
> > >     > "pg_basebackup will not not work in tar mode against 9.4 and older
> > >     servers,
> > >     >  as we have introduced a new protocol option in that mode."
> > >
> > >     AFAIK, pg_basebackup has never worked across versions.  So there's no
> > >     reason for this note.
> > >
> > >
> > > It has. The resulting backup has not been usable cross version, but
> > > pg_basebackup itself has. Though not always, and I'm not sure we've ever
> > > claimed it was supported, but it has worked.
> >
> > So we should still mention it in the release notes?
> >
> 
> If it has never lead to usable backup's for cross version backup, then I think
> there is no pressing need to mention it.

Removed.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Alvaro Herrera]

Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
> 
>   http://momjian.us/pgsql_docs/release-9-5.html

One more thing on names.  You have Alexander Shulgin as Alex Shulgin in
the second place his name appears.  Also, since he moved to Zalando (I
guess to Berlin) he writes his name as Oleksandr.  I guess he'd have to
tell us which forms he would like to be used in the release notes.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Alvaro Herrera]

Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
> 
>   http://momjian.us/pgsql_docs/release-9-5.html

I noticed something while reading this and would like the input of our
Japanese contributors.

Normally, western names are written using the given name first, then the
surname ("last name").  I am not well-versed on Japanese culture but my
understanding is that they use their family name first, then their given
name -- for instance, in "Yamada Tarō", Yamada is the family name and
Tarō is the given name.

Apparently, when interacting with Western people, some Japanese seem to
invert this and put the given name first.  Also, some write the family
name in all caps -- I guess this is done so that it is clear which part
is which.

Now, I think we should consider using a single style for all the
Japanese names used in the release notes.  Can we have our Japanese
contributors all agree on which style to use, and then let us know
what's their name in that style?

The names we currently have in the 9.5 release notes are:

Fujii Masao
Kyotaro Horiguchi
Sawada Masahiko
KaiGai Kohei
Shigeru Hanada
Etsuro Fujita
Furuya Osamu
MauMau
Mitsumasa Kondo


BTW, it is pretty cool to have contributor names that are natively in
scripts other than latin.  It is a pity that because of toolchain
limitations we cannot display names in kanji, cyrillic or other
character sets (in addition to their transliteration to latin script).


-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Peter Geoghegan]

On Thu, Jun 11, 2015 at 1:23 PM, Peter Geoghegan  wrote:
> I'm not sure if it's worth mentioning the "cheap equality for text"
> commit (e246b3d6eac09). I guess that it probably is, because it will
> help with things like index scans, too. Arguably that isn't a sorting
> thing (it's certainly not *just* a sorting thing).

I think we should really address this. Attached patch adds a new
release note item for it. It also adds to the documentation that
explains why users should prefer varchar(n)/text to character(n); the
lack of abbreviated key support now becomes a huge disadvantage for
character(n), whereas in previous versions the disadvantages were
fairly minor.

In passing, I updated the existing sort item to reflect that only
varchar(n), text, and numeric benefit from the abbreviation
optimization (not character types more generally + numeric), and added
a note on the effectiveness of the abbreviation optimization alone.

-- 
Peter Geoghegan
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 9d5ce95..bef9c6c 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -1106,7 +1106,7 @@ SELECT '52093.89'::money::numeric::float8;
 

 
- There is no performance difference among these three types,
+ There is generally no performance difference among these three types,
  apart from increased storage space when using the blank-padded
  type, and a few extra CPU cycles to check the length when storing into
  a length-constrained column.  While
@@ -1114,9 +1114,11 @@ SELECT '52093.89'::money::numeric::float8;
  advantages in some other database systems, there is no such advantage in
  PostgreSQL; in fact
  character(n) is usually the slowest of
- the three because of its additional storage costs.  In most situations
- text or character varying should be used
- instead.
+ the three because of its additional storage costs, and
+ particularly because presently it does not take advantage of an
+ important optimization used when sorting.  In most situations
+ text or character varying should be
+ used instead.
 

 
diff --git a/doc/src/sgml/release-9.5.sgml b/doc/src/sgml/release-9.5.sgml
index 279fb11..9c97b9b 100644
--- a/doc/src/sgml/release-9.5.sgml
+++ b/doc/src/sgml/release-9.5.sgml
@@ -214,8 +214,32 @@
 
   

-Improve the speed of sorting character and numeric fields (Peter
-Geoghegan, Andrew Gierth, Robert Haas)
+Improve the speed of sorting
+varchar(n), text,
+and numeric fields (Peter Geoghegan, Andrew
+Gierth, Robert Haas)
+   
+
+   
+When there are multiple fields, the optimization is most
+effective when most comparisons are resolved by comparing only
+the first field.
+   
+  
+
+  
+   
+Perform inexpensive binary equality pre-check for
+varchar(n) and text
+comparisons (Peter Geoghegan)
+   
+
+   
+This improves the speed of operations where many
+comparisons can be resolved with the pre-check, which may
+include sort operations, merge joins, and index scans.
+Multi-field sort operations with leading fields that have
+relatively few distinct values will particularly benefit.

   
 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] point_ops for GiST

[Emre Hasegeli]

> Emre Hasegeli just pointed out to me that this patch introduced
> box_contain_pt() and in doing so used straight C comparison (<= etc)
> instead of FPlt() and friends.  I would think that that's a bug and
> needs to be changed -- but certainly not backpatched, because gist
> indexes would/might become corrupt.

The problem with this is BRIN inclusion opclass uses some operators to
implement others.  It was using box @> point operator to implement
point ~= point operator by indexing points in boxes.  The former
doesn't use the macros, but later does.  The opclass could return
wrong result when the point right near the index boundaries.

Currently, there are not BRIN opclasses for geometric types except box
because of this reason.  I would like to work on supporting them for
the next release.  I think the best way is to change the operators
which are not using the macros to be consistent with the others.  Here
is the list:

* polygon << polygon
* polygon &< polygon
* polygon &> polygon
* polygon >> polygon
* polygon <<| polygon
* polygon &<| polygon
* polygon |&> polygon
* polygon |>> polygon
* box @> point
* point <@ box
* lseg <@ box
* circle @> point
* point <@ circle

I can send a patch, if it is acceptable.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Why does replication need the old history file?

[Josh Berkus]

>> The related source code comment says
>>
>> /*
>>  * Get any missing history files. We do this always, even when we're
>>  * not interested in that timeline, so that if we're promoted to
>>  * become the master later on, we don't select the same timeline that
>>  * was already used in the current master. This isn't bullet-proof -
>>  * you'll need some external software to manage your cluster if you
>>  * need to ensure that a unique timeline id is chosen in every case,
>>  * but let's avoid the confusion of timeline id collisions where we
>>  * can.
>>  */
>> WalRcvFetchTimeLineHistoryFiles(startpointTLI, primaryTLI);
> 
> So this seems to be something we're doing "just in case" which is
> preventing a useful way to spin up large master/replica clusters from
> PITR backup.  Might this be something we want to change, and simply
> warn that we can't find the history file instead of ERROR?

Having experimented with this further, it seems to me that either we
have a functionality bug here, or a documentation bug.  Currently our
docs do not treat the history files as critical, so much so that authors
of backup tools, like Barman, treat them as expireable.  But replication
apparently regards them as being as essiential as CLOG files.  The
result is that, in some cases, a Barman restored database can never
participate in replication again.

So either we should remove the requirement to have old history files to
start replication, *or* we should document somewhere to never, ever
delete history files and to retain them permanently for backups.  I know
what I'd prefer, but I'd like to hear the case for what could go wrong
if we don't retain the old history file requirement.

BTW, if retaining history files forever is a requirement, then maybe
they should go in their own directory in the future.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] On columnar storage

[Tom Lane]

Alvaro Herrera  writes:
> Tom Lane wrote:
>> I can't help thinking that this could tie in with the storage level API
>> that I was waving my arms about last year.  Or maybe not --- the goals
>> are substantially different --- but I think we ought to reflect on that
>> rather than just doing a narrow hack for column stores used in the
>> particular way you're describing here.

> I can't seem to remember this proposal you mention.  Care to be more
> specific?  Perhaps a link to archives is enough.

I never got to the point of having a concrete proposal, but there was a
discussion about it at last year's PGCon unconference; were you there?

Anyway the idea was to try to cut a clearer line between heap storage
and the upper levels of the system, particularly the catalog/DDL code
that we have so much of.  Based on Salesforce's experience so far,
it's near impossible to get rid of HeapTuple as the lingua franca for
representing rows in the upper system levels, so we've not really tried;
but it would be nice if the DDL code weren't so much in bed with
heap-specific knowledge, like the wired-into-many-places assumption that
row insert and update actions require index updates but deletions don't.
We're also not very happy with the general assumption that a TID is an
adequate row identifier (as our storage engine does not have TIDs), so
I'm a bit disappointed to see you doubling down on that restriction
rather than trying to lift it.

Now much of this pain only comes into play if one is trying to change
the underlying storage format for system catalogs, which I gather is
not considered in your proposal.  But if you want one format for catalogs
and another for user tables then you have issues like how do you guarantee
atomic commit and crash safety across multiple storage engines.  That way
lies a mess, especially if you're trying to keep the engines at arms'
length which is what a pluggable architecture implies.  MySQL is a
cautionary example we should be keeping in mind while thinking about this.

I don't really have any answers in this area, just questions.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] On columnar storage

[Alvaro Herrera]

Tom Lane wrote:

> I can't help thinking that this could tie in with the storage level API
> that I was waving my arms about last year.  Or maybe not --- the goals
> are substantially different --- but I think we ought to reflect on that
> rather than just doing a narrow hack for column stores used in the
> particular way you're describing here.

I can't seem to remember this proposal you mention.  Care to be more
specific?  Perhaps a link to archives is enough.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Alvaro Herrera]

Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
> 
>   http://momjian.us/pgsql_docs/release-9-5.html
>   
> and it will eventually appear here:
> 
>   http://www.postgresql.org/docs/devel/static/release.html
> 
> I am ready to make suggested adjustments, though I am traveling for
> conferences for the next ten days so there might a delay in my replies.

Many people have commented on the "RETURN WHERE" text.  The item in
question is this:
Have CREATE TABLE AS and REFRESH return an OID

This code was later changed to return ObjectAddress rather than OID;
this is used to feed event triggers.  To answer the "where" question:
the OID is returned to ProcessUtilitySlow, where it can be used by the
DDL-command-collection infrastructure.

I don't think this particular change needs a mention in the release
notes, since the ddl command collection is mentioned separately.  I
would just remove it.


In the pg_upgrade section, I think this is a bit strangely worded:
Remove pg_upgrade support for 8.3 old clusters (Bruce Momjian)
How about this?
Remove support for upgrading from 8.3 clusters (Bruce Momjian)

Why do we need a separate mention that pg_upgrade_support was moved to
backend in the "Additional Modules" section?  Seems to me that having
it mentioned in the pg_upgrade section would be better.


This entry is missing a word at the end:
Allow CURRENT/SESSION_USER to specify the current user in
certain

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 9.5 release notes

[Alvaro Herrera]

Petr Jelinek wrote:
> Hi,
> 
> +  
> +   
> +Add JSONB functions jsonb_set() and
> +jsonb_pretty (Dmitry Dolgov, Andrew Dunstan)
> +   
> +  
> 
> I believe I should be 3rd author for this one as I rewrote large parts of
> this functionality as part of the review.

Also, it looks like we could spell your last name with an accent,
assuming the i-acute character in Latin1 is fine.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Is it possible to have a "fast-write" Index?

[deavid]

Sorry; Because some misconfiugration vacuum and analyze were'nt working.
Now I'm getting better numbers for BRIN indexes where there are zero rows
to match.

El sáb., 13 jun. 2015 a las 3:17, deavid ()
escribió:

> So I just ran a test case for hash, btree, gin_btree and brin indexes.
> Also without indexes, and without primary keys.
> * Testing "deliverynotes" table.
>   - Definition and use case:
> It is a table contaning real delivery note headers of several years
> It consists of 300k rows, 128 columns, 63 indexes, 243Mb of data
> excluding indexes. Since is a table visible for users, almost every
> column can be searched so we need lots of indexes. We do not need
> searches to be the fastest possible, we only need to accelerate a
> bit our user searches; without harming too much writes.
>   - Things to test:
> - measure index creation times.
> - measure index space.
> - with indexes but without primary key
> - with everything
> - Create fully, delete everything and Insert again data in blocks
> - Test updates for recent data
>
> I attached the logs for every test, if anyone wants to see what i'm
> exactly testing.
> This was tested on my i5 laptop with 4Gb of RAM and a 120Gb SSD (OCZ
> Agility 3). I'm trying to measure CPU time, not I/O time, so some
> configurations and tests are specific to avoid as much as IO as I can.
> I'm using a dev build for Postgresql 9.5 downloaded from git sources.
>
> Conclusions:
> - Gin_btree seems slower in almost every case. It's writes are marginally
> better than regular btrees even when using work_mem=160MB. (May be 20%
> faster than btree). They are smaller than I thought.
> - BRIN indexes seem very fast for writes. For selects maybe is a blend
> between having indexes and don't having them. They don't recognize that
> some values are simply out of range of indexed values, and that's a pity.
> If the values we want are packed together I guess I would get even better
> results.
> - Primary keys and uniqueness checks doesn't seem to make any difference
> here.
> - Having no indexes at all is faster than I imagined. (Sometimes it beats
> BRIN or Btree) Maybe because the IO here is faster than usual.
> - Hash indexes: i tried to do something, but they take too much time to
> build and i don't know why. If creates are slow, updates should be slow
> too. I'm not going to test them again.
>
> And finally, don't know why but i couldn't vacuum or analyze tables. It
> always get stalled without doing anything; so i had to comment every
> vacuum. Maybe there is a bug in this dev version or i misconfigured
> something.
>
> El vie., 12 jun. 2015 a las 7:27, Simon Riggs ()
> escribió:
>
>> On 5 June 2015 at 18:07, deavid  wrote:
>>
>>> There are several use cases where I see useful an index, but adding it
>>> will slow too much inserts and updates.
>>> For example, when we have 10 million rows on a table, and it's a table
>>> which has frequent updates, we need several index to speed up selects, but
>>> then we'll slow down updates a lot, specially when we have 10 or more
>>> indexes.
>>> Other cases involve indexes for text search, which are used only for
>>> user search and aren't that important, so we want to have them, but we
>>> don't want the overload they put whenever we write on the table.
>>> I know different approaches that already solve some of those problems in
>>> some ways (table partitioning, partial indexes, etc), but i don't feel they
>>> are the solution to every problem of this kind.
>>>
>>> Some people already asked for "delayed write" indexes, but the idea gets
>>> discarded because the index could get out of sync, so it can omit results
>>> and this is unacceptable. But i think maybe that could be fixed in several
>>> ways and we can have a fast and reliable index (but maybe not so fast on
>>> selects).
>>>
>>
>> This is exactly the use case and mechanism for BRIN indexes.
>>
>> --
>> Simon Riggshttp://www.2ndQuadrant.com/
>> 
>>
>> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>>
>

Re: [HACKERS] Collection of memory leaks for ECPG driver

[Michael Meskes]

On Sat, Jun 13, 2015 at 12:02:40AM -0400, Tom Lane wrote:
> But having said that, I would not be in a hurry to remove any existing
> if-guards for the case.  For one thing, it makes the code look more
> similar to backend code that uses palloc/pfree, where we do *not* allow
> pfree(NULL).  There's also the point that changing longstanding code
> creates back-patching hazards, so unless there's a clear gain it's best
> not to.

Makes sense, but there is no point in adding hos if-guards to old code that
doesn't have it either,right?

Michael
-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Meskes at (Debian|Postgresql) dot Org
Jabber: michael.meskes at gmail dot com
VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers