[HACKERS] Storing pg_stat_statements query texts externally, pg_stat_statements in core

2013-11-14 Thread Peter Geoghegan
Attached patch allows pg_stat_statements to store arbitrarily long query texts, using an external, transparently managed lookaside file. This is of great practical benefit to certain types of users, who need to understand the execution costs of queries with associated excessively long query

Re: [HACKERS] nested hstore patch

2013-11-14 Thread Hannu Krosing
On 11/14/2013 01:32 AM, David E. Wheeler wrote: On Nov 13, 2013, at 3:59 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I remember strong voices in support of *not* normalising json, so that things like {a:1,a:true, a:b, a:none} would go through the system unaltered, for claimed standard

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-14 Thread David Rowley
On Sun, Nov 3, 2013 at 3:18 AM, David Rowley dgrowle...@gmail.com wrote: I'm low on ideas on how to improve things much around here for now, but for what it's worth, I did create a patch which changes unnecessary calls to appendPQExpBuffer() into calls to appendPQExpBufferStr() similar to the

Re: [HACKERS] [PATCH] pg_basebackup: progress report max once per second

2013-11-14 Thread Mika Eloranta
On 13 Nov 2013, at 20:51, Mika Eloranta m...@ohmu.fi wrote: Prevent excessive progress reporting that can grow to gigabytes of output with large databases. Same patch as an attachment. -- Mika Eloranta Ohmu Ltd. http://www.ohmu.fi/

Re: [HACKERS] GIN improvements part 1: additional information

2013-11-14 Thread Alexander Korotkov
On Tue, Nov 5, 2013 at 9:49 PM, Heikki Linnakangas hlinnakan...@vmware.comwrote: On 04.11.2013 23:44, Alexander Korotkov wrote: On Mon, Oct 21, 2013 at 11:12 PM, Alexander Korotkov aekorot...@gmail.comwrote: Attached version of patch is debugged. I would like to note that number of bugs

Re: [HACKERS] tcp_keepalives_idle

2013-11-14 Thread Marko Tiikkaja
On 11/14/13 7:08 AM, Tatsuo Ishii wrote: It means the connection is idle except for keepalive packets. We could perhaps just drop the word otherwise, if people find it confusing. Wah. I seemed to completely misunderstand what the pharase says. Thanks for clarification. I agree to drop

Re: [HACKERS] [PATCH] pg_basebackup: progress report max once per second

2013-11-14 Thread Marko Tiikkaja
On 11/14/13 10:27 AM, Mika Eloranta wrote: On 13 Nov 2013, at 20:51, Mika Eloranta m...@ohmu.fi wrote: Prevent excessive progress reporting that can grow to gigabytes of output with large databases. Same patch as an attachment. I can't comment on the usefulness of this patch, but the first

Re: [HACKERS] Extra functionality to createuser

2013-11-14 Thread Sameer Thakur
Hello, Tried to test this patch. Did the following 1. cloned from https://github.com/samthakur74/postgres 2. Applied patch and make install 3. created rolesapp_readonly_role,app2_writer_role 4. Tried createuser -D -S -l -g app_readonly_role,app2_writer_role test_user got error: createuser: invalid

Re: [HACKERS] [GENERAL] Clang 3.3 Analyzer Results

2013-11-14 Thread Magnus Hagander
On Wednesday, November 13, 2013, Tom Lane wrote: Kevin Grittner kgri...@ymail.com javascript:; writes: If nobody objects, I'll fix that small memory leak in the regression test driver. Hopefully someone more familiar with pg_basebackup will fix the double-free (and related problems

Re: [HACKERS] [PATCH 1/2] SSL: GUC option to prefer server cipher order

2013-11-14 Thread Magnus Hagander
On Thursday, November 7, 2013, Marko Kreen wrote: On Wed, Nov 06, 2013 at 09:57:32PM -0300, Alvaro Herrera wrote: Marko Kreen escribió: By default OpenSSL (and SSL/TLS in general) lets client cipher order take priority. This is OK for browsers where the ciphers were tuned, but few

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-14 Thread Florian Weimer
On 11/14/2013 07:02 AM, Sawada Masahiko wrote: I attached patch adds new wal_level 'all'. Shouldn't this be a separate setting? It's useful for storage which requires rewriting a partially written sector before it can be read again. -- Florian Weimer / Red Hat Product Security Team --

Re: [HACKERS] Re: Using indexes for ORDER BY and PARTITION BY clause in windowing functions

2013-11-14 Thread Kyotaro HORIGUCHI
Hello, When I read it again and try to relate, I get your point. Actually true, hashes must always be performed as last option (if that is what you too meant) and if there are few other operations they must be the last one to be performed especially after sorting/grouping. Hashes must somehow

Re: [HACKERS] Extra functionality to createuser

2013-11-14 Thread Sameer Thakur
1. cloned from https://github.com/samthakur74/postgres Sorry. cloned from https://github.com/postgres/postgres regards Sameer -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GIN improvements part 1: additional information

2013-11-14 Thread Alexander Korotkov
On Thu, Nov 14, 2013 at 2:17 PM, Alexander Korotkov aekorot...@gmail.comwrote: On Tue, Nov 5, 2013 at 9:49 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 04.11.2013 23:44, Alexander Korotkov wrote: On Mon, Oct 21, 2013 at 11:12 PM, Alexander Korotkov aekorot...@gmail.comwrote:

Re: [HACKERS] Improve code in tidbitmap.c

2013-11-14 Thread Etsuro Fujita
I'd like to do the complementary explanation of this. In tbm_union_page() and tbm_intersect_page() in tidbitmap.c, WORDS_PER_PAGE is used in the scan of a lossy chunk, instead of WORDS_PER_CHUNK, where these macros are defined as: /* number of active words for an exact page: */ #define

[HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-14 Thread Haribabu kommi
Please find attached the patch, for adding a new option for pg_basebackup, to specify a different directory for pg_xlog. Design A new option: xlogdir is added to the list of options for pg_basebackup. The new option is not having an equivalent short option letter. This option will allow the

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-11-14 Thread KONDO Mitsumasa
(2013/10/21 20:17), KONDO Mitsumasa wrote: (2013/10/18 22:21), Andrew Dunstan wrote: If we're going to extend pg_stat_statements, even more than min and max I'd like to see the standard deviation in execution time. OK. I do! I am making some other patches, please wait more! I add stddev_time

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-11-14 Thread KONDO Mitsumasa
Oh! Sorry... I forgot to attach my latest patch. Regards, -- Mitsumasa KONDO NTT Open Source Software Center diff --git a/contrib/pg_stat_statements/pg_stat_statements--1.1--1.2.sql b/contrib/pg_stat_statements/pg_stat_statements--1.1--1.2.sql new file mode 100644 index 000..929d623 ---

Re: [HACKERS] nested hstore patch

2013-11-14 Thread Andrew Dunstan
On 11/14/2013 03:21 AM, Hannu Krosing wrote: On 11/14/2013 01:32 AM, David E. Wheeler wrote: On Nov 13, 2013, at 3:59 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I remember strong voices in support of *not* normalising json, so that things like {a:1,a:true, a:b, a:none} would go through

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread Claudio Freire
On Thu, Nov 14, 2013 at 9:09 AM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I create a patch that is improvement of disk-read and OS file caches. It can optimize kernel readahead parameter using buffer access strategy and posix_fadvice() in various disk-read situations. In general

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread Heikki Linnakangas
On 14.11.2013 14:38, David Rowley wrote: I've just completed some more benchmarking of this. I didn't try dropping the threshold down to 2 or 0 but I did tests at the cut over point and really don't see much difference in performance between the list at 32 and the hashtable at 33 sequences. The

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread Andres Freund
Hi, On 2013-11-13 22:55:43 +1300, David Rowley wrote: Here http://www.postgresql.org/message-id/24278.1352922...@sss.pgh.pa.us there was some talk about init_sequence being a bottleneck when many sequences are used in a single backend. The attached I think implements what was talked about

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I think it'd be a better idea to integrate the sequence caching logic into the relcache. There's a comment about it: * (We can't * rely on the relcache, since it's only, well, a cache, and may decide to * discard entries.) but that's not really

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread Andres Freund
On 2013-11-14 09:23:20 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I think it'd be a better idea to integrate the sequence caching logic into the relcache. There's a comment about it: * (We can't * rely on the relcache, since it's only, well, a cache, and may

Re: [HACKERS] [GENERAL] Clang 3.3 Analyzer Results

2013-11-14 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: That code was originally stolen from psql, and then whacked around a number of times. The part about looping and passwords, for example, is in startup.c in psql as well. We probably want to fix it there as well (even if it doesn't have the same

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-11-14 09:23:20 -0500, Tom Lane wrote: We most certainly *do* discard entries, if they're not open when a cache flush event comes along. What I was aiming at is that we don't discard them because of a limited cache size. I don't think it

Re: [HACKERS] additional json functionality

2013-11-14 Thread Garick Hamlin
On Wed, Nov 13, 2013 at 04:50:49PM -0800, David E. Wheeler wrote: On Nov 13, 2013, at 4:45 PM, Andrew Dunstan and...@dunslane.net wrote: It should be a pretty-printing function option, perhaps, but not core to the type itself, IMO. I don't in the least understand how it could be a

Re: [HACKERS] [PATCH] ecpg: Split off mmfatal() from mmerror()

2013-11-14 Thread Michael Meskes
On Tue, Nov 12, 2013 at 10:22:20PM -0500, Peter Eisentraut wrote: Similar to recent pg_upgrade changes (https://commitfest.postgresql.org/action/patch_view?id=1216), here is a patch to separate the terminating and nonterminating variants of mmerror() in ecpg. ... Haven't tried it, but it

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread Andres Freund
On 2013-11-14 09:47:18 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-11-14 09:23:20 -0500, Tom Lane wrote: We most certainly *do* discard entries, if they're not open when a cache flush event comes along. What I was aiming at is that we don't discard them

[HACKERS] Somebody broke \d on indexes

2013-11-14 Thread Tom Lane
In HEAD: regression=# \d tenk1_thous_tenthous ERROR: column i.indisidentity does not exist LINE 4: i.indisidentity, ^ This works fine in released versions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] additional json functionality

2013-11-14 Thread Merlin Moncure
On Wed, Nov 13, 2013 at 6:01 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 11/14/2013 12:09 AM, Merlin Moncure wrote: On Wed, Nov 13, 2013 at 4:16 PM, Josh Berkus j...@agliodbs.com wrote: On 11/13/2013 06:45 AM, Merlin Moncure wrote: I'm not so sure we should require hstore to do things

Re: [HACKERS] Somebody broke \d on indexes

2013-11-14 Thread Andres Freund
On 2013-11-14 09:52:11 -0500, Tom Lane wrote: In HEAD: regression=# \d tenk1_thous_tenthous ERROR: column i.indisidentity does not exist LINE 4: i.indisidentity, ^ That's me. At some point indisidentity was renamed to indisreplident. Patch attached (also renaming a variable that

Re: [HACKERS] Somebody broke \d on indexes

2013-11-14 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-11-14 09:52:11 -0500, Tom Lane wrote: In HEAD: regression=# \d tenk1_thous_tenthous ERROR: column i.indisidentity does not exist LINE 4: i.indisidentity, ^ That's me. At some point indisidentity was renamed to indisreplident. Patch

Re: [HACKERS] additional json functionality

2013-11-14 Thread Hannu Krosing
On 11/14/2013 12:20 AM, Josh Berkus wrote: Merlin, I use pg/JSON all over the place. In several cases I have to create documents with ordered keys because the parser on the other side wants them that way -- this is not a hypothetical argument. The current json serialization API handles

Re: [HACKERS] additional json functionality

2013-11-14 Thread Hannu Krosing
On 11/14/2013 01:42 AM, Andrew Dunstan wrote: On 11/13/2013 07:01 PM, Hannu Krosing wrote: I guess we should not replace current JSON type with hstore based one, but add something json-like based on nested hstore instead. Well, that's two voices for that course of action. I am not really

Re: [HACKERS] additional json functionality

2013-11-14 Thread Merlin Moncure
On Thu, Nov 14, 2013 at 9:42 AM, Hannu Krosing ha...@2ndquadrant.com wrote: This is supported by the fact that current functions on json-source treat it as json-object (for example key lookup gives you the value of latest key and not a list of all matching key values). yeah. hm. that's a good

[HACKERS] Ideas of printing out the alternative paths

2013-11-14 Thread Zhan Li
When searching all the possible paths of executing a query, the optimizer finds and saves the cheapest paths for the top level rel. I'd like to check out all the paths the optimizer has ever considered, which I believe, are stored in the pathlist of the top level rel. But I do not have an idea of

Re: [HACKERS] additional json functionality

2013-11-14 Thread Hannu Krosing
On 11/14/2013 04:07 PM, Merlin Moncure wrote: On Wed, Nov 13, 2013 at 6:01 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I guess we should not replace current JSON type with hstore based one, but add something json-like based on nested hstore instead. Maybe call it jsdoc or jdoc or jsobj or

Re: [HACKERS] nested hstore patch

2013-11-14 Thread Hannu Krosing
On 11/14/2013 01:47 PM, Andrew Dunstan wrote: On 11/14/2013 03:21 AM, Hannu Krosing wrote: On 11/14/2013 01:32 AM, David E. Wheeler wrote: On Nov 13, 2013, at 3:59 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I remember strong voices in support of *not* normalising json, so that things

Re: [HACKERS] additional json functionality

2013-11-14 Thread Hannu Krosing
On 11/14/2013 05:06 PM, Merlin Moncure wrote: On Thu, Nov 14, 2013 at 9:42 AM, Hannu Krosing ha...@2ndquadrant.com wrote: This is supported by the fact that current functions on json-source treat it as json-object (for example key lookup gives you the value of latest key and not a list of all

Re: [HACKERS] Ideas of printing out the alternative paths

2013-11-14 Thread Tom Lane
Zhan Li zhanl...@gmail.com writes: When searching all the possible paths of executing a query, the optimizer finds and saves the cheapest paths for the top level rel. I'd like to check out all the paths the optimizer has ever considered, which I believe, are stored in the pathlist of the top

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread Fujii Masao
On Thu, Nov 14, 2013 at 9:09 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Hi, I create a patch that is improvement of disk-read and OS file caches. It can optimize kernel readahead parameter using buffer access strategy and posix_fadvice() in various disk-read situations. When I

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-11-14 Thread Fujii Masao
On Thu, Nov 14, 2013 at 7:11 AM, Peter Geoghegan p...@heroku.com wrote: On Wed, Oct 23, 2013 at 8:52 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Hmm, now if we had portable atomic addition, so that we could spare the spinlock ... That certainly seems like an interesting possibility.

Re: [HACKERS] Ideas of printing out the alternative paths

2013-11-14 Thread Zhan Li
Thank you for your reply Tom. Then a) what are exactly stored in the pathlist of top level rel? Paths worth considering? b) I have been struggling to come up with a way to print the Path struct. If I can print a path the way like A hash join (B nested loop join C), that would be great. You

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-14 Thread Alexander Korotkov
On Sun, Jun 30, 2013 at 3:00 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 28.06.2013 22:31, Alexander Korotkov wrote: Now, I got the point of three state consistent: we can keep only one consistent in opclasses that support new interface. exact true and exact false values will

Re: [HACKERS] additional json functionality

2013-11-14 Thread Merlin Moncure
On Thu, Nov 14, 2013 at 10:54 AM, Hannu Krosing ha...@2ndquadrant.com wrote: On 11/14/2013 05:06 PM, Merlin Moncure wrote: On Thu, Nov 14, 2013 at 9:42 AM, Hannu Krosing ha...@2ndquadrant.com wrote: This is supported by the fact that current functions on json-source treat it as json-object

[HACKERS] AWS RDS now supports PostgreSQL!

2013-11-14 Thread Rayson Ho
http://aws.typepad.com/aws/2013/11/amazon-rds-for-postgresql-now-available.html (The Free Tier page has not been updated yet, but I believe PostgreSQL should also be free for new AWS users: http://aws.amazon.com/free/ ) Rayson == Open Grid

Re: [HACKERS] additional json functionality

2013-11-14 Thread David E. Wheeler
On Nov 14, 2013, at 7:07 AM, Merlin Moncure mmonc...@gmail.com wrote: This is exactly what needs to be done, full stop (how about: hstore). It really comes down to this: changing the serialization behaviors that have been in production for 2 releases (three if you count the extension) is bad

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-14 Thread Fujii Masao
On Thu, Nov 14, 2013 at 9:08 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: Please find attached the patch, for adding a new option for pg_basebackup, to specify a different directory for pg_xlog. Sounds good! Here are the review comments: +printf(_(--xlogdir=XLOGDIR

Re: [HACKERS] LISTEN / NOTIFY enhancement request for Postgresql

2013-11-14 Thread Bruce Momjian
On Thu, Oct 24, 2013 at 11:41:57AM -0400, Sev Zaslavsky wrote: Here is an example implementation: http://activemq.apache.org/nms/ activemq-wildcards.html • is used to separate names in a path • * is used to match any name in a path • is used to recursively match any destination

Re: [HACKERS] AWS RDS now supports PostgreSQL!

2013-11-14 Thread Bruce Momjian
On Thu, Nov 14, 2013 at 12:29:58PM -0500, Rayson Ho wrote: http://aws.typepad.com/aws/2013/11/amazon-rds-for-postgresql-now-available.html (The Free Tier page has not been updated yet, but I believe PostgreSQL should also be free for new AWS users: http://aws.amazon.com/free/ ) Here is the

Re: [HACKERS] Additional information on log_line_prefix

2013-11-14 Thread Bruce Momjian
On Fri, Oct 25, 2013 at 02:58:12PM -0400, Andrew Dunstan wrote: On 10/25/2013 01:50 PM, Emanuel Calvo wrote: Hi guys, I'm working on a quick convertion script for query reviews and I wonder if a feature request to add the following values will be possible: %D = duration %L =

Re: [HACKERS] additional json functionality

2013-11-14 Thread Merlin Moncure
On Thu, Nov 14, 2013 at 11:34 AM, David E. Wheeler da...@justatheory.com wrote: On Nov 14, 2013, at 7:07 AM, Merlin Moncure mmonc...@gmail.com wrote: This is exactly what needs to be done, full stop (how about: hstore). It really comes down to this: changing the serialization behaviors that

Re: [HACKERS] Ideas of printing out the alternative paths

2013-11-14 Thread Tom Lane
Zhan Li zhanl...@gmail.com writes: Thank you for your reply Tom. Then a) what are exactly stored in the pathlist of top level rel? Paths worth considering? b) I have been struggling to come up with a way to print the Path struct. If I can print a path the way like A hash join (B nested loop

Re: [HACKERS] additional json functionality

2013-11-14 Thread Hannu Krosing
On 11/14/2013 08:17 PM, Merlin Moncure wrote: On Thu, Nov 14, 2013 at 11:34 AM, David E. Wheeler da...@justatheory.com wrote: On Nov 14, 2013, at 7:07 AM, Merlin Moncure mmonc...@gmail.com wrote: This is exactly what needs to be done, full stop (how about: hstore). It really comes down to

Re: [HACKERS] additional json functionality

2013-11-14 Thread David Johnston
Hannu Krosing-5 wrote On 11/14/2013 08:17 PM, Merlin Moncure wrote: On Thu, Nov 14, 2013 at 11:34 AM, David E. Wheeler lt; david@ gt; wrote: On Nov 14, 2013, at 7:07 AM, Merlin Moncure lt; mmoncure@ gt; wrote: This is exactly what needs to be done, full stop (how about: hstore). It

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-14 Thread Heikki Linnakangas
On 14.11.2013 19:26, Alexander Korotkov wrote: On Sun, Jun 30, 2013 at 3:00 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 28.06.2013 22:31, Alexander Korotkov wrote: Now, I got the point of three state consistent: we can keep only one consistent in opclasses that support new

Re: [HACKERS] LISTEN / NOTIFY enhancement request for Postgresql

2013-11-14 Thread Dimitri Fontaine
Bruce Momjian br...@momjian.us writes: • is used to separate names in a path • * is used to match any name in a path • is used to recursively match any destination starting from this name For example using the example above, these subscriptions are possible Subscription

Re: [HACKERS] Transaction-lifespan memory leak with plpgsql DO blocks

2013-11-14 Thread Tom Lane
I wrote: Robert Haas robertmh...@gmail.com writes: I'm not volunteering to spend time fixing this, but I disagree with the premise that it isn't worth fixing, because I think it's a POLA violation. Yeah, I'm not terribly comfortable with letting it go either. Attached is a proposed patch.

Re: [HACKERS] Assertions in PL/PgSQL

2013-11-14 Thread Pavel Stehule
rebased patch Regards Pavel 2013/11/14 Peter Eisentraut pete...@gmx.net On Wed, 2013-10-09 at 18:57 +0200, Pavel Stehule wrote: here is a patch for RAISE WHEN clause Your patch needs to be rebased. diff --git a/doc/src/sgml/plpgsql.sgml b/doc/src/sgml/plpgsql.sgml index

Re: [HACKERS] Extra functionality to createuser

2013-11-14 Thread Christopher Browne
On Thu, Nov 14, 2013 at 5:41 AM, Sameer Thakur samthaku...@gmail.com wrote: So i think -g option is failing Right you are. I was missing a g: in the getopt_long() call. Attached is a revised patch that handles that. And it behaves better: postgres@cbbrowne ~/p/s/b/scripts ./createuser -g

[HACKERS] union all query consumes all memory

2013-11-14 Thread Pavel Stehule
Hello, one my customer reported a out of memory issue. After investigation he found a main problem in large query that uses a lot of union all queries. He wrote a self test: do $$ declare i integer; str text=''; begin for i in 1..1000 loop str := str || 'union all select i,i,i from

Re: [HACKERS] union all query consumes all memory

2013-11-14 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: one my customer reported a out of memory issue. After investigation he found a main problem in large query that uses a lot of union all queries. He wrote a self test: do $$ declare i integer; str text=''; begin for i in 1..1000 loop str

[HACKERS] strncpy is not a safe version of strcpy

2013-11-14 Thread David Rowley
Hi All, As a bit of a background task, over the past few days I've been analysing the uses of strncpy in the code just to try and validate if it is the right function to be using. I've already seen quite a few places where their usage is wrongly assumed. As many of you will know and maybe some

[HACKERS] Anybody using get_eclass_for_sort_expr in an extension?

2013-11-14 Thread Tom Lane
I looked into bug #8591: http://www.postgresql.org/message-id/e1vgk41-00050x...@wrigleys.postgresql.org and was able to reproduce the problem. The proximate cause is that get_eclass_for_sort_expr is wrong to suppose that it can always create new equivalence class entries with empty

[HACKERS] SSL: better default ciphersuite

2013-11-14 Thread Marko Kreen
Attached patch changes the default ciphersuite to HIGH:!aNULL instead of old DEFAULT:!LOW:!EXP:!MD5:@STRENGTH where DEFAULT is a shortcut for ALL:!aNULL:!eNULL. Main goal is to leave low-level ciphersuite details to OpenSSL guys and give clear impression to Postgres admins what it is

Re: [HACKERS] Anybody using get_eclass_for_sort_expr in an extension?

2013-11-14 Thread Peter Geoghegan
On Thu, Nov 14, 2013 at 3:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm a bit inclined to take the risk of breaking anything that's calling get_eclass_for_sort_expr() directly. Thoughts? It's worth being aware of the fact that Peter E's Jenkins instance seems to track regressions for some

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread David Rowley
On Fri, Nov 15, 2013 at 3:03 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 14.11.2013 14:38, David Rowley wrote: I've just completed some more benchmarking of this. I didn't try dropping the threshold down to 2 or 0 but I did tests at the cut over point and really don't see much

Re: [HACKERS] strncpy is not a safe version of strcpy

2013-11-14 Thread Tomas Vondra
On 15 Listopad 2013, 0:07, David Rowley wrote: Hi All, As a bit of a background task, over the past few days I've been analysing the uses of strncpy in the code just to try and validate if it is the right function to be using. I've already seen quite a few places where their usage is

Re: [HACKERS] strncpy is not a safe version of strcpy

2013-11-14 Thread David Rowley
On Fri, Nov 15, 2013 at 12:33 PM, Tomas Vondra t...@fuzzy.cz wrote: It is likely far better explained here -- http://www.courtesan.com/todd/papers/strlcpy.html For example , the following 2 lines in jsonfuncs.c memset(name, 0, NAMEDATALEN); strncpy(name, fname, NAMEDATALEN); Be

[HACKERS] Review: Patch insert throw error when year field len 4 for timestamptz datatype

2013-11-14 Thread Adrian Klaver
Initial review of the patch submitted in this message and listed in the current CommitFest: http://www.postgresql.org/message-id/cagpqqf3xwwc_4fhinz_g6ecvps_ov3k2pe4-aj1dg4iyy+f...@mail.gmail.com This patch would seem to be already committed here

Re: [HACKERS] Review: Patch insert throw error when year field len 4 for timestamptz datatype

2013-11-14 Thread Peter Geoghegan
On Thu, Nov 14, 2013 at 4:53 PM, Adrian Klaver adrian.kla...@gmail.com wrote: Is a review necessary at this point? No. Just mark it committed. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread David Rowley
On Fri, Nov 15, 2013 at 3:12 AM, Andres Freund and...@2ndquadrant.comwrote: Hi, On 2013-11-13 22:55:43 +1300, David Rowley wrote: Here http://www.postgresql.org/message-id/24278.1352922...@sss.pgh.pa.usthere was some talk about init_sequence being a bottleneck when many sequences are

Re: [HACKERS] Review: Patch insert throw error when year field len 4 for timestamptz datatype

2013-11-14 Thread Bruce Momjian
On Thu, Nov 14, 2013 at 04:58:55PM -0800, Peter Geoghegan wrote: On Thu, Nov 14, 2013 at 4:53 PM, Adrian Klaver adrian.kla...@gmail.com wrote: Is a review necessary at this point? No. Just mark it committed. Oh, sorry, I didn't realize it was in the commit-fest app. -- Bruce Momjian

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread KONDO Mitsumasa
Hi Claudio, (2013/11/14 22:53), Claudio Freire wrote: On Thu, Nov 14, 2013 at 9:09 AM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I create a patch that is improvement of disk-read and OS file caches. It can optimize kernel readahead parameter using buffer access strategy and

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread KONDO Mitsumasa
(2013/11/15 2:03), Fujii Masao wrote: On Thu, Nov 14, 2013 at 9:09 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Hi, I create a patch that is improvement of disk-read and OS file caches. It can optimize kernel readahead parameter using buffer access strategy and posix_fadvice() in

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread Peter Geoghegan
On Thu, Nov 14, 2013 at 6:18 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I will fix it. Could you tell me your Mac OS version and gcc version? I have only mac book air with Maverick OS(10.9). I have an idea that Mac OSX doesn't have posix_fadvise at all. Didn't you use the relevant

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-11-14 Thread KONDO Mitsumasa
(2013/11/14 7:11), Peter Geoghegan wrote: On Wed, Oct 23, 2013 at 8:52 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Hmm, now if we had portable atomic addition, so that we could spare the spinlock ... And adding a histogram or min/max for something like execution time isn't an approach

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-11-14 Thread Peter Geoghegan
On Thu, Nov 14, 2013 at 9:09 AM, Fujii Masao masao.fu...@gmail.com wrote: I think that pg_stat_statements should be made to do this kind of thing by a third party tool that aggregates snapshots of deltas. Time-series data, including (approximate) *local* minima and maxima should be built from

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread KONDO Mitsumasa
(2013/11/15 11:17), Peter Geoghegan wrote: On Thu, Nov 14, 2013 at 6:18 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I will fix it. Could you tell me your Mac OS version and gcc version? I have only mac book air with Maverick OS(10.9). I have an idea that Mac OSX doesn't have

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-11-14 Thread Peter Geoghegan
On Thu, Nov 14, 2013 at 6:28 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: It is confirmation just to make sure, does this patch mean my patch? I agree with you about not adding another lock implementation. It will becomes overhead. Yes, I referred to your patch. I don't want to go

Re: [HACKERS] information schema parameter_default implementation

2013-11-14 Thread Peter Eisentraut
Updated patch attached. On Sat, 2013-11-09 at 12:09 +0530, Amit Khandekar wrote: 2) I found the following check a bit confusing, maybe you can make it better if (!argmodes || argmodes[i] == PROARGMODE_IN || argmodes[i] == PROARGMODE_INOUT || argmodes[i] == PROARGMODE_VARIADIC)

[HACKERS] Database disconnection and switch inside a single bgworker

2013-11-14 Thread Michael Paquier
Hi all, Currently, bgworkers offer the possibility to connect to a given database using BackgroundWorkerInitializeConnection in bgworker.h, but there is actually no way to disconnect from a given database inside the same bgworker process. One of the use cases for that would be the possibility to

Re: [HACKERS] Logging WAL when updating hintbit

2013-11-14 Thread Sawada Masahiko
On Thu, Nov 14, 2013 at 3:53 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Nov 14, 2013 at 3:02 PM, Sawada Masahiko sawada.m...@gmail.com wrote: I attached patch adds new wal_level 'all'. If wal_level is set 'all', the server logs WAL not only when wal_level is set

[HACKERS] Autoconf 2.69 update

2013-11-14 Thread Peter Eisentraut
I'm proposing that we upgrade our Autoconf to 2.69, which is the latest right now (release date 2012-04-24). There are no changes in the source needed, just tweak the version number in configure.in (see below) and run autoreconf. I've compared the configure output before and after on a few

Re: [HACKERS] Autoconf 2.69 update

2013-11-14 Thread Michael Paquier
On Fri, Nov 15, 2013 at 12:00 PM, Peter Eisentraut pete...@gmx.net wrote: I'm proposing that we upgrade our Autoconf to 2.69, which is the latest right now (release date 2012-04-24). There are no changes in the source needed, just tweak the version number in configure.in (see below) and run

Re: [HACKERS] strncpy is not a safe version of strcpy

2013-11-14 Thread Tomas Vondra
On 15 Listopad 2013, 1:00, David Rowley wrote: On Fri, Nov 15, 2013 at 12:33 PM, Tomas Vondra t...@fuzzy.cz wrote: It is likely far better explained here -- http://www.courtesan.com/todd/papers/strlcpy.html For example , the following 2 lines in jsonfuncs.c memset(name, 0,

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-11-14 Thread Peter Geoghegan
On Tue, Nov 5, 2013 at 5:30 AM, Sameer Thakur samthaku...@gmail.com wrote: Hello, Please find attached pg_stat_statements-identification-v9.patch. I took a quick look. Observations: + /* Making query ID dependent on PG version */ + query-queryId |= PG_VERSION_NUM 16; If you want

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-11-14 Thread KONDO Mitsumasa
(2013/11/15 11:31), Peter Geoghegan wrote: On Thu, Nov 14, 2013 at 6:28 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: It is confirmation just to make sure, does this patch mean my patch? I agree with you about not adding another lock implementation. It will becomes overhead. Yes, I

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2013-11-14 Thread KONDO Mitsumasa
(2013/11/15 2:09), Fujii Masao wrote: Agreed. Could you tell me your agreed reason? I am sorry that I suspect you doesn't understand this disccusion enough:-( Regards, -- Mitsumasa KONDO NTT Open Source Software Ceter -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-14 Thread Amit Kapila
On Wed, Nov 13, 2013 at 12:02 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 12 November 2013 08:47 Amit Kapila wrote: On Mon, Nov 11, 2013 at 3:14 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 08 November 2013 18:35 Amit Kapila wrote: On Fri, Nov 8, 2013 at 10:56 AM,

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-11-14 Thread Daniel Farina
On Thu, Nov 14, 2013 at 7:25 PM, Peter Geoghegan p...@heroku.com wrote: On Tue, Nov 5, 2013 at 5:30 AM, Sameer Thakur samthaku...@gmail.com wrote: Hello, Please find attached pg_stat_statements-identification-v9.patch. I took a quick look. Observations: + /* Making query ID dependent

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread Claudio Freire
On Thu, Nov 14, 2013 at 11:13 PM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: Hi Claudio, (2013/11/14 22:53), Claudio Freire wrote: On Thu, Nov 14, 2013 at 9:09 AM, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I create a patch that is improvement of disk-read and OS file

Re: [HACKERS] Heavily modified big table bloat even in auto vacuum is running

2013-11-14 Thread Haribabu kommi
On 15 November 2013 10:00 Amit Kapila wrote: On Wed, Nov 13, 2013 at 12:02 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 12 November 2013 08:47 Amit Kapila wrote: On Mon, Nov 11, 2013 at 3:14 PM, Haribabu kommi haribabu.ko...@huawei.com wrote: On 08 November 2013 18:35 Amit

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread David Rowley
On Fri, Nov 15, 2013 at 3:03 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote I think that means that we should just completely replace the list with the hash table. The difference with a small N is lost in noise, so there's no point in keeping the list as a fast path for small N. That'll

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-11-14 Thread Pavel Stehule
+1 interesting feature Pavel 2013/11/15 Peter Eisentraut pete...@gmx.net Various places in the constraint checking code say something like, if we ever implement assertions, here is where it should go. I've been fiddling with filling in those gaps for some time now, and the other day I

Re: [HACKERS] init_sequence spill to hash table

2013-11-14 Thread David Rowley
On Fri, Nov 15, 2013 at 3:23 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: I think it'd be a better idea to integrate the sequence caching logic into the relcache. There's a comment about it: * (We can't * rely on the relcache, since it's only,

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-11-14 Thread Sameer Thakur
I took a quick look. Observations: + /* Making query ID dependent on PG version */ + query-queryId |= PG_VERSION_NUM 16; If you want to do something like this, make the value of PGSS_FILE_HEADER incorporate (PG_VERSION_NUM / 100) or something. Why are you doing this? The thought was

Re: [HACKERS] GIN improvements part 1: additional information

2013-11-14 Thread Alexander Korotkov
On Thu, Nov 14, 2013 at 3:00 PM, Alexander Korotkov aekorot...@gmail.comwrote: On Thu, Nov 14, 2013 at 2:17 PM, Alexander Korotkov aekorot...@gmail.comwrote: On Tue, Nov 5, 2013 at 9:49 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 04.11.2013 23:44, Alexander Korotkov wrote:

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-14 Thread Alexander Korotkov
On Fri, Nov 15, 2013 at 3:25 AM, Rod Taylor r...@simple-knowledge.comwrote: I checked out master and put together a test case using a small percentage of production data for a known problem we have with Pg 9.2 and text search scans. A small percentage in this case means 10 million records

  1   2   >