Re: [HACKERS] RangeType internal use

2015-02-05 Thread Amit Langote
Horiguchi-san, On 06-02-2015 PM 04:34, Kyotaro HORIGUCHI wrote: > Hi, from nearby:) > Thank you! >> I wonder why I cannot find a way to get a range type for a given (sub-) >> type. I would like to build a RangeType from Datum's of lower and upper >> bounds. Much like how construct_array() buil

Re: [HACKERS] RangeType internal use

2015-02-05 Thread Kyotaro HORIGUCHI
Hi, from nearby:) > I wonder why I cannot find a way to get a range type for a given (sub-) > type. I would like to build a RangeType from Datum's of lower and upper > bounds. Much like how construct_array() builds an ArrayType from a Datum > array of elements given elements' type info. > > Is th

Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2015-02-05 Thread Michael Paquier
On Fri, Jan 30, 2015 at 10:26 PM, Andreas Karlsson wrote: > On 01/30/2015 07:48 AM, Michael Paquier wrote: >> Looking at the latest patch, it seems that in >> AlterTableGetLockLevel@tablecmds.c we ought to put AT_ReAddConstraint, >> AT_AddConstraintRecurse and AT_ProcessedConstraint under the same

Re: [HACKERS] Early Setup of instrumentation information in pg_stat_statements

2015-02-05 Thread Amit Kapila
On Thu, Feb 5, 2015 at 8:58 PM, Amit Kapila wrote: > > Currently in pg_stat_statements, the setup to track > instrumentation/totaltime information is done after > ExecutorStart(). Can we do this before ExecutorStart()? On further analyzing, I found that currently it is done after ExecutorStart()

Re: [HACKERS] pg_basebackup may fail to send feedbacks.

2015-02-05 Thread Kyotaro HORIGUCHI
Sorry, I misunderstood that. > > At Wed, 4 Feb 2015 19:22:39 +0900, Fujii Masao > > wrote in > > > >> On Wed, Feb 4, 2015 at 4:58 PM, Kyotaro HORIGUCHI > >> wrote: > >> > I'm very sorry for confused report. The problem found in 9.4.0 > >> > and the diagnosis was mistakenly done on master. > >

Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2015-02-05 Thread Michael Paquier
On Thu, Feb 5, 2015 at 11:58 PM, Michael Paquier wrote: > An updated patch is attached. I just noticed that the patch I sent was incorrect: - Parameter name was still wal_availability_check_interval and not wal_retrieve_retry_interval - Documentation was incorrect. Please use the patch attached ins

[HACKERS] pg_basebackup, tablespace mapping and path canonicalization

2015-02-05 Thread Ian Barwick
Hi I stumbled on what appears to be inconsistent handling of double slashes in tablespace paths when using pg_basebackup with the -T/--tablespace-mapping option: ibarwick:postgresql (master)$ mkdir /tmp//foo-old ibarwick:postgresql (master)$ $PG_HEAD/psql 'dbname=postgres port=9595' p

Re: [HACKERS] Table-level log_autovacuum_min_duration

2015-02-05 Thread Michael Paquier
On Fri, Feb 6, 2015 at 4:50 AM, Naoya Anzai wrote: >> The following review has been posted through the commitfest application: >> make installcheck-world: tested, failed >> Implements feature: tested, failed >> Spec compliant: tested, failed >> Documentation:tested, fa

[HACKERS] No-rewrite timestamp<->timestamptz conversions

2015-02-05 Thread Noah Misch
On Tue, Nov 05, 2013 at 05:02:58PM -0800, Josh Berkus wrote: > I'd also love some way of doing a no-rewrite conversion between > timestamp and timestamptz, based on the assumption that the original > values are UTC time. That's one I encounter a lot. It was such a conversion that motivated me to

Re: [HACKERS] Table-level log_autovacuum_min_duration

2015-02-05 Thread Naoya Anzai
> The following review has been posted through the commitfest application: > make installcheck-world: tested, failed > Implements feature: tested, failed > Spec compliant: tested, failed > Documentation:tested, failed I'm sorry, I just sent it by mistake. All of them ha

[HACKERS] RangeType internal use

2015-02-05 Thread Amit Langote
Hi, I wonder why I cannot find a way to get a range type for a given (sub-) type. I would like to build a RangeType from Datum's of lower and upper bounds. Much like how construct_array() builds an ArrayType from a Datum array of elements given elements' type info. Is there some way I do not seem

Re: [HACKERS] Table-level log_autovacuum_min_duration

2015-02-05 Thread Naoya Anzai
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:tested, failed Hi, I'm Naoya Anzai. I've been working as a PostgreSQL Suppo

[HACKERS] HEADS UP: PGCon 2015 major schedule changes

2015-02-05 Thread Dan Langille
Hello, By request, the format of PGCon 2015 will differ significantly from previous year. Our goal is to give you more of what you want while still keeping the stuff you've always liked. In June 2015, PGCon will be structured as follows: Unconference: 16-17 June 2015 (Tue afternoon & all day W

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread David Steele
On 2/5/15 4:53 PM, Josh Berkus wrote: >> Actually, perhaps we should have a boolean setting that just implies >> min=max, instead of having a configurable minimum?. That would cover all >> of those reasons pretty well. So we would have a "max_wal_size" setting, >> and a boolean "preallocate_all_wal

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
On 02/05/2015 01:42 PM, Heikki Linnakangas wrote: > There are a few reasons for making the minimum configurable: Any thoughts on what the default minimum should be, if the default max is 1.1GB/64? > 1. Creating new segments when you need them is not free, so if you have > a workload with occasion

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Robert Haas
>>> Default of 4 for min_wal_size? >> >> I assume you mean 4 segments; why not 3 as currently? As long as the >> system has the latitude to ratchet it up when needed, there seems to >> be little advantage to raising the minimum. Of course I guess there >> must be some advantage or Heikki wouldn't

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Robert Haas
On Thu, Feb 5, 2015 at 4:42 PM, Heikki Linnakangas wrote: > Actually, perhaps we should have a boolean setting that just implies > min=max, instead of having a configurable minimum?. That would cover all of > those reasons pretty well. So we would have a "max_wal_size" setting, and a > boolean "pr

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Heikki Linnakangas
On 02/05/2015 11:28 PM, Robert Haas wrote: On Thu, Feb 5, 2015 at 2:11 PM, Josh Berkus wrote: Default of 4 for min_wal_size? I assume you mean 4 segments; why not 3 as currently? As long as the system has the latitude to ratchet it up when needed, there seems to be little advantage to raisin

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
On 02/05/2015 01:28 PM, Robert Haas wrote: > On Thu, Feb 5, 2015 at 2:11 PM, Josh Berkus wrote: >> Except that, when setting up servers for customers, one thing I pretty >> much always do for them is temporarily increase checkpoint_segments for >> the initial data load. So having Postgres do this

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Robert Haas
On Thu, Feb 5, 2015 at 2:11 PM, Josh Berkus wrote: > Except that, when setting up servers for customers, one thing I pretty > much always do for them is temporarily increase checkpoint_segments for > the initial data load. So having Postgres do this automatically would > be a feature, not a bug.

Re: [HACKERS] SSL renegotiation and other related woes

2015-02-05 Thread Heikki Linnakangas
On 01/26/2015 12:14 PM, Andres Freund wrote: Hi, When working on getting rid of ImmediateInterruptOK I wanted to verify that ssl still works correctly. Turned out it didn't. But neither did it in master. Turns out there's two major things we do wrong: 1) We ignore the rule that once called and

Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2015-02-05 Thread Michael Paquier
Fujii Masao wrote: > +TimestampDifference(start_time, stop_time, &secs, µsecs); > +pg_usleep(interval_msec * 1000L - (100L * secs + 1L * microsecs)); > > What if the large interval is set and a signal arrives during the sleep? > I'm afraid that a signal cannot terminate the sleep on som

Re: [HACKERS] s_lock.h default definitions are rather confused

2015-02-05 Thread Andres Freund
On 2015-01-15 17:59:40 +0100, Andres Freund wrote: > On 2015-01-15 11:56:24 -0500, Tom Lane wrote: > > Andres Freund writes: > > > On 2015-01-15 10:57:10 -0500, Tom Lane wrote: > > >> While I'll not cry too hard when we decide to break C89 compatibility, > > >> I don't want it to happen accidental

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-05 Thread Michael Paquier
On Thu, Feb 5, 2015 at 11:06 PM, Syed, Rahila wrote: >>/* >>+* We recheck the actual size even if pglz_compress() report success, >>+* because it might be satisfied with having saved as little as one byte >>+* in the compressed data. >>+*/ >>+ *len = (uint16) compressed_len; >>+

Re: [HACKERS] Possible problem with pgcrypto

2015-02-05 Thread Jan Wieck
On 02/05/2015 01:18 PM, Marko Tiikkaja wrote: On 2/5/15 4:48 PM, Jan Wieck wrote: What the script does is to encode a small string with pgp_sym_encrypt() and then repeatedly try to decrypt it with different "wrong" passwords. The expected error message for that is of course "Wrong key or

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
On 02/04/2015 04:16 PM, David Steele wrote: > On 2/4/15 3:06 PM, Robert Haas wrote: >>> Hmmm, I see your point. I spend a lot of time on AWS and in >>> container-world, where disk space is a lot more constrained. However, >>> it probably makes more sense to recommend non-default settings for that

Re: [HACKERS] Possible problem with pgcrypto

2015-02-05 Thread Jan Wieck
On 02/05/2015 10:58 AM, Tom Lane wrote: Jan Wieck writes: I have encountered a small instability in the behavior of pgcrypto's pgp_sym_decrypt() function. Attached is a script that can reproduce the problem. It may have to be run repeatedly because the symptom occurs rather seldom. What the

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-05 Thread Michael Paquier
Fujii Masao wrote: > I wrote >> This is an inspiration from lz4 APIs. Wouldn't it be buggy for a >> compression algorithm to return a size of 0 bytes as compressed or >> decompressed length btw? We could as well make it return a negative >> value when a failure occurs if you feel more comfortable w

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Whether this is a realistic expectation given the wording of the SQL-MED > standard is unclear. I've only got a draft as of 2011-12 handy, but I'm not seeing anything in the standard that cares one bit about the value of the options specified for the FDW.

Re: [HACKERS] Possible problem with pgcrypto

2015-02-05 Thread Marko Tiikkaja
On 2/5/15 4:48 PM, Jan Wieck wrote: What the script does is to encode a small string with pgp_sym_encrypt() and then repeatedly try to decrypt it with different "wrong" passwords. The expected error message for that is of course "Wrong key or corrupt data". Every now and then, I get a dif

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-05 Thread Tom Lane
Robert Haas writes: > We've got a mix of styles for extensible options right now: That we do. > So COPY puts the options at the very end, but EXPLAIN and VACUUM put > them right after the command name. I prefer the latter style and > would vote to adopt it here. Meh. Options-at-the-end seems

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-05 Thread Robert Haas
On Wed, Feb 4, 2015 at 8:24 PM, Tom Lane wrote: > Kyotaro HORIGUCHI writes: >> The phrase "{INDEX | TABLE |..} name" seems to me indivisible as >> target specification. IMHO, the options for VACUUM and so is >> placed *just after* command name, not *before* the target. > >> If this is right, the

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread David G Johnston
Tom Lane-2 wrote > Stephen Frost < > sfrost@ > > writes: >> * Robert Haas ( > robertmhaas@ > ) wrote: >>> On Thu, Feb 5, 2015 at 10:48 AM, Stephen Frost < > sfrost@ > > wrote: And I thought this was about FDW options and not about dblink, really.. > >>> The OP is pretty clearly asking a

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Tom Lane
Stephen Frost writes: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Thu, Feb 5, 2015 at 10:48 AM, Stephen Frost wrote: >>> And I thought this was about FDW options and not about dblink, really.. >> The OP is pretty clearly asking about dblink. > I was just pointing out that it was an iss

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Feb 5, 2015 at 10:48 AM, Stephen Frost wrote: > >> If she's got dblink access, she can run arbitrary > >> SQL queries on the remote server anyway, which is all the password > >> would let her do. Also, she could use dblink to run ALTER ROLE f

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Robert Haas
On Thu, Feb 5, 2015 at 10:48 AM, Stephen Frost wrote: >> If she's got dblink access, she can run arbitrary >> SQL queries on the remote server anyway, which is all the password >> would let her do. Also, she could use dblink to run ALTER ROLE foo >> PASSWORD '...' on the remote server, and then s

Re: [HACKERS] Possible problem with pgcrypto

2015-02-05 Thread Tom Lane
Jan Wieck writes: > I have encountered a small instability in the behavior of pgcrypto's > pgp_sym_decrypt() function. Attached is a script that can reproduce the > problem. It may have to be run repeatedly because the symptom occurs > rather seldom. > What the script does is to encode a small

[HACKERS] Possible problem with pgcrypto

2015-02-05 Thread Jan Wieck
Hi, I have encountered a small instability in the behavior of pgcrypto's pgp_sym_decrypt() function. Attached is a script that can reproduce the problem. It may have to be run repeatedly because the symptom occurs rather seldom. What the script does is to encode a small string with pgp_sym_e

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Feb 3, 2015 at 6:26 PM, Noah Yetter wrote: > > The obvious objection is, "well you should just use foreign tables instead > > of dblink()". I'll cut a long story short by saying that doesn't work for > > us. We are using postgres_fdw to allo

[HACKERS] Early Setup of instrumentation information in pg_stat_statements

2015-02-05 Thread Amit Kapila
Currently in pg_stat_statements, the setup to track instrumentation/totaltime information is done after ExecutorStart(). Can we do this before ExecutorStart()? In particular, I am referring to below code: static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags) { .. standard_ExecutorStart

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Tom Lane
Robert Haas writes: > All that having been said, it wouldn't be crazy to try to invent a > system to lock this down, but it *would* be complicated. An > individual FDW can call its authentication-related options anything it > likes; they do not need to be called 'password'. So we'd need a way >

Re: [HACKERS] [GENERAL] 4B row limit for CLOB tables

2015-02-05 Thread Matthew Kelly
> That's assuming that toasting is evenly spread between tables. In my > experience, that's not a great bet... Time to create a test: SELECT chunk_id::bigint/10 as id_range, count(*), count(*)/(10::float) density FROM (SELECT chunk_id FROM pg_toast.pg_toast_39000165 WHERE chunk_id < 100

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Robert Haas
On Tue, Feb 3, 2015 at 6:26 PM, Noah Yetter wrote: > The obvious objection is, "well you should just use foreign tables instead > of dblink()". I'll cut a long story short by saying that doesn't work for > us. We are using postgres_fdw to allow our analysts to run queries against > AWS Redshift

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Heikki Linnakangas
On 02/05/2015 04:47 PM, Andres Freund wrote: On 2015-02-05 09:42:37 -0500, Robert Haas wrote: I previously proposed 100 segments, or 1.6GB. If that seems too large, how about 64 segments, or 1.024GB? I think there will be few people who can't tolerate a gigabyte of xlog under peak load, and an

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Andres Freund
On 2015-02-05 09:42:37 -0500, Robert Haas wrote: > I previously proposed 100 segments, or 1.6GB. If that seems too > large, how about 64 segments, or 1.024GB? I think there will be few > people who can't tolerate a gigabyte of xlog under peak load, and an > awful lot who will benefit from it. It

[HACKERS] Simplify sleeping while reading/writing from client

2015-02-05 Thread Heikki Linnakangas
Looking again at the code after Andres' interrupt-handling patch series, I got confused by the fact that there are several wait-retry loops in different layers, and reading and writing works slightly differently. I propose the attached, which pulls all the wait-retry logic up to secure_read()

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Robert Haas
On Wed, Feb 4, 2015 at 4:41 PM, Josh Berkus wrote: >> That's certainly better, but I think we should go further. Again, >> you're not committed to using this space all the time, and if you are >> using it you must have a lot of write activity, which means you are >> not running on a tin can and a

Re: [HACKERS] Unlikely-to-happen crash in ecpg driver caused by NULL-pointer check not done

2015-02-05 Thread Michael Meskes
On Tue, Feb 03, 2015 at 10:44:17PM +0900, Michael Paquier wrote: > All those things are addressed in the patch attached. Fixed a typo and commited. Thanks Michael for fixing and Heikki for reviewing. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-05 Thread Syed, Rahila
Hello, >/* >+* We recheck the actual size even if pglz_compress() report success, >+* because it might be satisfied with having saved as little as one byte >+* in the compressed data. >+*/ >+ *len = (uint16) compressed_len; >+ if (*len >= orig_len - 1) >+ return false; >

Re: [HACKERS] File based Incremental backup v9

2015-02-05 Thread Francesco Canovai
Hi Marco, On Sunday 01 February 2015 00:47:24 Marco Nenciarini wrote: > You can find the updated patch attached to this message. I've been testing the v9 patch with checksums enabled and I end up with a lot of warnings like these ones: WARNING: page verification failed, calculated checksum 473

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-05 Thread Fujii Masao
On Tue, Jan 6, 2015 at 11:09 AM, Michael Paquier wrote: > On Mon, Jan 5, 2015 at 10:29 PM, Fujii Masao wrote: >> On Sun, Dec 28, 2014 at 10:57 PM, Michael Paquier wrote: >> The patch 1 cannot be applied to the master successfully because of >> recent change. > Yes, that's caused by ccb161b. Attac