Re: [HACKERS] pg_basebackup is taking backup of extra files inside a tablespace directory

2012-11-28 Thread Hari Babu
On Wed, Nov 28, 2012 at 12:43 PM, Michael Paquier michael(dot)paquier(at)gmail(dot)comwrote: On Wed, Nov 28, 2012 at 3:55 PM, Hari Babu haribabu(dot)kommi(at)huawei(dot)comwrote: pg_basebackup is taking backup of extra files other than database related files in side a TABLESPACE directory.

[HACKERS] Strange behaviour with incompatible psql/server

2012-11-28 Thread Pavan Deolasee
A friend reported this issue to me and I find it a bit strange and even after spending some time on this, I couldn't really figure out what's going wrong. See attached two SQL files, bad.sql and good.sql. They look the exact same in the editor. In fact, the good.sql is created by copying lines

Re: [HACKERS] Strange behaviour with incompatible psql/server

2012-11-28 Thread Heikki Linnakangas
On 28.11.2012 10:46, Pavan Deolasee wrote: While I'm almost certain that this has something to do with special characters that my naked eyes can not see, all my attempts to spot the difference has failed. So I really have two questions: 1. What's the difference between these files ? Compare

Re: [HACKERS] Strange behaviour with incompatible psql/server

2012-11-28 Thread Pavan Deolasee
On Wed, Nov 28, 2012 at 2:28 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 28.11.2012 10:46, Pavan Deolasee wrote: While I'm almost certain that this has something to do with special characters that my naked eyes can not see, all my attempts to spot the difference has failed. So

Re: [HACKERS] Strange behaviour with incompatible psql/server

2012-11-28 Thread Pavan Deolasee
On Wed, Nov 28, 2012 at 2:52 PM, Pavan Deolasee pavan.deola...@gmail.comwrote: Should this not be back patched ? The error that's coming because not having this fix is rather very strange and hard to debug for any average individual. I'd almost concluded that one should NEVER use an old psql

Re: [HACKERS] plpgsql_check_function - rebase for 9.3

2012-11-28 Thread Pavel Stehule
Hello a some updated version * possibility to raise (and filter) performance warnings - detects IO castings * detects assign composite value to scalar variable Regards Pavel Stehule 2012/11/27 Pavel Stehule pavel.steh...@gmail.com: Hello I am sending a updated version - now it is prepared

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2012-11-28 Thread Amit Kapila
On Wednesday, November 28, 2012 11:49 AM Muhammad Usama wrote: On Tue, Nov 27, 2012 at 5:52 PM, Amit kapila amit.kap...@huawei.com wrote: Friday, November 23, 2012 5:38 AM Muhammad Usama wrote:   - For -p {file | dir}  option the utility expects the file path relative to the specified data

Re: [HACKERS] FDW for PostgreSQL

2012-11-28 Thread Kohei KaiGai
2012/11/28 Shigeru Hanada shigeru.han...@gmail.com: On Sun, Nov 25, 2012 at 5:24 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: I checked the v4 patch, and I have nothing to comment anymore. So, could you update the remaining EXPLAIN with VERBOSE option stuff? Thanks for the review. Here

Re: [HACKERS] FDW for PostgreSQL

2012-11-28 Thread Kohei KaiGai
2012/11/28 Kohei KaiGai kai...@kaigai.gr.jp: it is reasonable. So, postgre_fdw is OK for me. pgsql_fdw is also welcome. Sorry, s/postgre_fdw/postgres_fdw/g Thanks, -- KaiGai Kohei kai...@kaigai.gr.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Andres Freund
On 2012-11-27 23:46:58 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-11-27 16:31:15 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Isn't inisprimary updated when an ALTER TABLE ... ADD PRIMARY KEY ... USING someindex ; is done? Also I

Re: [HACKERS] Enabling frontend-only xlog desc routines

2012-11-28 Thread Amit Kapila
On Wednesday, November 28, 2012 12:17 AM Alvaro Herrera wrote: I mentioned the remaining issues in a previous email (see message-id 20121025161751.ge6...@alvh.no-ip.org). Attached is a patch that enables xlogdump to #include xlog_internal.h by way of removing that file's inclusion of fmgr.h,

Re: [HACKERS] Enabling frontend-only xlog desc routines

2012-11-28 Thread Andres Freund
On 2012-11-28 18:58:45 +0530, Amit Kapila wrote: On Wednesday, November 28, 2012 12:17 AM Alvaro Herrera wrote: I mentioned the remaining issues in a previous email (see message-id 20121025161751.ge6...@alvh.no-ip.org). Attached is a patch that enables xlogdump to #include xlog_internal.h

[HACKERS] Review: Extra Daemons / bgworker

2012-11-28 Thread Markus Wanner
Hello Álvaro, first of all, thank you for bringing this up again and providing a patch. My first attempt on that was more than two years ago [1]. As the author of a former bgworker patch, I'd like to provide an additional review - KaiGai was simply faster to sing up as a reviewer on the

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-28 Thread Bruce Momjian
On Tue, Nov 27, 2012 at 09:35:10PM -0800, Jeff Janes wrote: On Tue, Nov 27, 2012 at 8:13 PM, Bruce Momjian br...@momjian.us wrote: I have some new interesting results (in seconds, test script attached): -Fc --- dump | pg_restore/psql -- - pg_upgrade -

[HACKERS] pg_upgrade test picks up unwanted environment values

2012-11-28 Thread Andrew Dunstan
I noticed yesterday while doing some buildfarm tinkering that the test script for pg_upgrade doesn't unset various important environment values as pg_regress does. This can cause it to fail e.g. is PGUSER is set, as it was with my testing. I propose to add this to the script unless there are

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-28 Thread Alvaro Herrera
Markus Wanner wrote: Hi Markus, Many thanks for your review. first of all, thank you for bringing this up again and providing a patch. My first attempt on that was more than two years ago [1]. As the author of a former bgworker patch, I'd like to provide an additional review - KaiGai was

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-28 Thread Markus Wanner
On 11/28/2012 03:51 PM, Alvaro Herrera wrote: I remember your patchset. I didn't look at it for this round, for no particular reason. I did look at KaiGai's submission from two commitfests ago, and also at a patch from Simon which AFAIK was never published openly. Simon's patch merged the

[HACKERS] json accessors

2012-11-28 Thread Andrew Dunstan
This is a proposal to create some basic functions to extract values from json. The simple functions I envision would be: * json_object_keys(json) = setof text returns the set of dequoted, unescaped keys of the object, errors if it's not an object * json_get(json, keytext) = json

[HACKERS] InvokeObjectAccessHook versus DROP INDEX CONCURRENTLY

2012-11-28 Thread Tom Lane
Just a side note that the above combination doesn't work, at least not if the object access hook tries to make any database state updates. I've put a hack into index_drop that should detect the case: /* * We must commit our transaction in order to make the first pg_index

Re: [HACKERS] InvokeObjectAccessHook versus DROP INDEX CONCURRENTLY

2012-11-28 Thread Alvaro Herrera
Tom Lane wrote: Just a side note that the above combination doesn't work, at least not if the object access hook tries to make any database state updates. I've put a hack into index_drop that should detect the case: /* * We must commit our transaction in order to make the

Re: [HACKERS] pg_dump --split patch

2012-11-28 Thread Alvaro Herrera
Marko Tiikkaja wrote: On 16/11/2012 15:52, Dimitri Fontaine wrote: What happens if you have a table foo and another table FoO? They would go to the same file. If you think there are technical issues behind that decision (e.g. the dump would not restore), I would like to hear an example

Re: [HACKERS] json accessors

2012-11-28 Thread Merlin Moncure
On Wed, Nov 28, 2012 at 11:04 AM, Andrew Dunstan and...@dunslane.net wrote: This is a proposal to create some basic functions to extract values from json. The simple functions I envision would be: * json_object_keys(json) = setof text returns the set of dequoted, unescaped keys of the

Re: [HACKERS] Materialized views WIP patch

2012-11-28 Thread Robert Haas
On Tue, Nov 27, 2012 at 10:58 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I unlike using keywords DO for this purpose - when we use it for anonymous blocks Yeah, I don't much like that either. My original suggestion when Kevin and I discussed this over voice was ALTER MATERIALIZED VIEW

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Andres Freund
On 2012-11-28 14:09:11 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-11-27 23:46:58 -0500, Tom Lane wrote: Attached is a very preliminary draft patch for this. I've not addressed the question of whether we can clear indcheckxmin during transactional updates

Re: [HACKERS] [PATCH] binary heap implementation

2012-11-28 Thread Andres Freund
On 2012-11-27 11:56:41 -0500, Robert Haas wrote: [ Sorry for the slow response on this, Thanksgiving interfered. ] On Wed, Nov 21, 2012 at 3:41 PM, Andres Freund and...@2ndquadrant.com wrote: One very minor nitpick I unfortunately just found now, not sure when that changed:

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-28 Thread Bruce Momjian
On Tue, Nov 27, 2012 at 09:35:10PM -0800, Jeff Janes wrote: I tested custom format with pg_restore -j and -1, as well as text restore. The winner was pg_dump -Fc | pg_restore -1; I don't have the numbers at hand, but if my relcache patch is accepted, then -1 stops being faster. -1 gets

Re: [HACKERS] Materialized views WIP patch

2012-11-28 Thread Kevin Grittner
Robert Haas wrote: I don't particularly like syntaxes involving DO or LOAD because those words already have strong associations with completely unrelated features. Now, if we don't want to do that and we don't want to use ALTER for a data-modifying command either, another option would be to

Re: [HACKERS] json accessors

2012-11-28 Thread Andrew Dunstan
On 11/28/2012 02:08 PM, Merlin Moncure wrote: On Wed, Nov 28, 2012 at 11:04 AM, Andrew Dunstan and...@dunslane.net wrote: This is a proposal to create some basic functions to extract values from json. The simple functions I envision would be: * json_object_keys(json) = setof text

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-28 Thread Alvaro Herrera
Merlin Moncure escribió: Maybe abstracting 'last xid cache' along with hint bit management out of both transam.c and tqual.c into something like 'hints.c' is appropriate, but that's a more invasive change. It would be good to have such a patch to measure/compare performance of both

Re: [HACKERS] json accessors

2012-11-28 Thread Andrew Dunstan
On 11/28/2012 03:44 PM, Andrew Dunstan wrote: As for json_to_hstore, as I mentioned, the design is intended to enable the easy constructyion of such transformations, although for hstores anything except trivial json structure (i.e. an unnested object) it might have unappealing results. But

Re: [HACKERS] foreign key locks

2012-11-28 Thread Kevin Grittner
Alvaro Herrera wrote: Here's version 24. This no longer applies after the rmgr rm_desc patch. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: One minor thing I haven't noticed earlier: Perhaps we should also skip over invalid indexes in transformTableLikeClause's CREATE_TABLE_LIKE_INDEXES case? I left that as-is intentionally: the fact that an index isn't valid doesn't prevent us from

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Andres Freund
On 2012-11-28 17:42:18 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: One minor thing I haven't noticed earlier: Perhaps we should also skip over invalid indexes in transformTableLikeClause's CREATE_TABLE_LIKE_INDEXES case? I left that as-is intentionally: the fact

Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules

2012-11-28 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 11/28/2012 02:14 PM, Alvaro Herrera wrote: Okapi has been failing sporadically on ecpg, and I wonder if it's related to this change. Well, it looks like the make is broken and missing a clear dependency requirement. I think we need to ask Jeremy

Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules

2012-11-28 Thread Peter Eisentraut
On 11/28/12 6:01 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 11/28/2012 02:14 PM, Alvaro Herrera wrote: Okapi has been failing sporadically on ecpg, and I wonder if it's related to this change. Well, it looks like the make is broken and missing a clear dependency

Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules

2012-11-28 Thread Andrew Dunstan
On 11/28/2012 06:01 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 11/28/2012 02:14 PM, Alvaro Herrera wrote: Okapi has been failing sporadically on ecpg, and I wonder if it's related to this change. Well, it looks like the make is broken and missing a clear dependency

Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules

2012-11-28 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 11/28/12 6:01 PM, Tom Lane wrote: I wonder whether adding another .NOTPARALLEL directive would be a better idea than insisting people get hold of patched versions. We could put ifeq ($(MAKE_VERSION),3.82) .NOTPARALLEL: endif into

Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules

2012-11-28 Thread Andrew Dunstan
On 11/28/2012 06:19 PM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On 11/28/12 6:01 PM, Tom Lane wrote: I wonder whether adding another .NOTPARALLEL directive would be a better idea than insisting people get hold of patched versions. We could put ifeq ($(MAKE_VERSION),3.82)

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2012-11-28 17:42:18 -0500, Tom Lane wrote: I agree it's a judgment call, though. Anybody want to argue for the other position? Hm. Seems odd to include indexes that are being dropped concurrently at that moment. But then, we can't really detect

Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules

2012-11-28 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 11/28/2012 06:19 PM, Tom Lane wrote: It appears to me that the case that okapi is hitting is specific to the ecpg preprocessor build rules, and indeed specific to the case where preproc.c needs to be rebuilt. A .NOTPARALLEL in

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Andres Freund
On 2012-11-28 18:41:39 -0500, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2012-11-28 17:42:18 -0500, Tom Lane wrote: I agree it's a judgment call, though. Anybody want to argue for the other position? Hm. Seems odd to include indexes that are being dropped concurrently

Re: [HACKERS] json accessors

2012-11-28 Thread Merlin Moncure
On Wed, Nov 28, 2012 at 2:44 PM, Andrew Dunstan and...@dunslane.net wrote: On 11/28/2012 02:08 PM, Merlin Moncure wrote: *) ISTM your keytext operators are a reasonable replacement for a hypothetical json_path. That said you're basically forcing json-sql mapping through a highly iterative

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2012-11-28 18:41:39 -0500, Tom Lane wrote: However, this is more complicated and harder to understand. So unless somebody is really excited about being able to tell the difference between create-in-progress and drop-in-progress, I'd rather leave

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Andres Freund
On 2012-11-29 09:10:22 +0900, Michael Paquier wrote: On Thu, Nov 29, 2012 at 8:52 AM, Andres Freund and...@2ndquadrant.comwrote: On 2012-11-28 18:41:39 -0500, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2012-11-28 17:42:18 -0500, Tom Lane wrote: I agree it's a

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Andres Freund
On 2012-11-28 19:11:46 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-11-28 18:41:39 -0500, Tom Lane wrote: However, this is more complicated and harder to understand. So unless somebody is really excited about being able to tell the difference between

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2012-11-29 09:10:22 +0900, Michael Paquier wrote: and is going to need a lot of rework as well as more infrastructure like a better MVCC-ish SnapshotNow. Which is a major project in itself. I wonder whether my crazy follow updates via t_ctid

Re: [HACKERS] json accessors

2012-11-28 Thread Hannu Krosing
On 11/29/2012 01:10 AM, Merlin Moncure wrote: On Wed, Nov 28, 2012 at 2:44 PM, Andrew Dunstan and...@dunslane.net wrote: ... *) have you considered something like anyelement from_json(anyelement, json) or select json::some_type; (this may or many not be possible given our casting mechanics;

Re: [HACKERS] json accessors

2012-11-28 Thread Hannu Krosing
On 11/29/2012 02:07 AM, Hannu Krosing wrote: On 11/29/2012 01:10 AM, Merlin Moncure wrote: On Wed, Nov 28, 2012 at 2:44 PM, Andrew Dunstan and...@dunslane.net wrote: ... *) have you considered something like anyelement from_json(anyelement, json) or select json::some_type; (this may or

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Michael Paquier
On 2012/11/29, at 9:23, Andres Freund and...@2ndquadrant.com wrote: On 2012-11-28 19:11:46 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-11-28 18:41:39 -0500, Tom Lane wrote: However, this is more complicated and harder to understand. So unless somebody is

Re: [HACKERS] [COMMITTERS] pgsql: Refactor flex and bison make rules

2012-11-28 Thread Tom Lane
Jeremy Drake pgbuildf...@jdrake.com writes: While we're talking about odd issues that only seem to happen on Okapi, does anyone know of anything I can do to diagnose the pg_upgrade failure on the 9.2 branch? There are no rogue (non-buildfarm-related) postmaster/postgres processes running on

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-28 Thread Pavan Deolasee
On Thu, Nov 29, 2012 at 10:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: Attached is an updated patch for HEAD that I think is about ready to go. I'll start making a back-patchable version shortly. Here is an only-lightly-tested version for 9.2. Looks good at a glance. I wonder

Re: [HACKERS] Materialized views WIP patch

2012-11-28 Thread Pavel Stehule
2012/11/28 Kevin Grittner kgri...@mail.com: Robert Haas wrote: I don't particularly like syntaxes involving DO or LOAD because those words already have strong associations with completely unrelated features. Now, if we don't want to do that and we don't want to use ALTER for a data-modifying

Re: [HACKERS] Enabling frontend-only xlog desc routines

2012-11-28 Thread Amit Kapila
On Wednesday, November 28, 2012 7:07 PM Andres Freund wrote: On 2012-11-28 18:58:45 +0530, Amit Kapila wrote: On Wednesday, November 28, 2012 12:17 AM Alvaro Herrera wrote: I mentioned the remaining issues in a previous email (see message-id 20121025161751.ge6...@alvh.no-ip.org).