Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread David E. Wheeler
On Sep 25, 2011, at 9:58 PM, Itagaki Takahiro wrote: I'd like to support UTF-8 text or csv files that has BOM (byte order mark) in COPY FROM command. BOM will be automatically detected and ignored if the file encoding is UTF-8. WIP patch attached. By my reading of

Re: [HACKERS] Remastering using streaming only replication?

2011-09-26 Thread Fujii Masao
On Thu, Sep 22, 2011 at 1:52 AM, Josh Berkus j...@agliodbs.com wrote: Fujii, I haven't really been following your latest patches about taking backups from the standby and cascading replication, but I wanted to see if it fulfills another TODO: the ability to remaster (that is, designate the

Re: [HACKERS] TABLE tab completion

2011-09-26 Thread Magnus Hagander
On Sun, Sep 25, 2011 at 15:06, Dean Rasheed dean.a.rash...@gmail.com wrote: On 24 September 2011 11:59, Magnus Hagander mag...@hagander.net wrote: TABLE tab completion in psql only completes to tables, not views. but the TABLE command works fine for both tables and views (and also sequences).

Re: [HACKERS] new createuser option for replication role

2011-09-26 Thread Fujii Masao
On Fri, Sep 23, 2011 at 10:47 PM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: 2011/9/23 Robert Haas robertmh...@gmail.com: On Thu, Sep 22, 2011 at 12:45 PM, Fujii Masao masao.fu...@gmail.com wrote: Agreed. Attached is the updated version of the patch. It adds two options

Re: [HACKERS] Online base backup from the hot-standby

2011-09-26 Thread Jun Ishiduka
Attached is the updated version of the patch. I refactored the code, fixed some bugs, added lots of source code comments, improved the document, but didn't change the basic design. Please check this patch, and let's use this patch as the base if you agree with that. Thanks for update patch.

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Kohei KaiGai
2011/9/26 Tom Lane t...@sss.pgh.pa.us: As a stopgap, what about removing sepgsql from the list of contrib modules tested by make -C contrib check?  (I haven't looked at exactly how ugly it might be to do that, nor whether we'd have to also disable installcheck from recursing to sepgsql.) Is

[HACKERS] bug of recovery?

2011-09-26 Thread Fujii Masao
Hi, Currently, if a reference to an invalid page is found during recovery, its information is saved in hash table invalid_page_tab. Then, if such a reference is resolved, its information is removed from the hash table. If there is unresolved reference to an invalid page in the hash table at the

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Kohei KaiGai
2011/9/26 Robert Haas robertmh...@gmail.com: On Sun, Sep 25, 2011 at 3:25 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: I'm a bit nervous about storing security_barrier in the RTE.  What happens to stored rules if the security_barrier option gets change later? The rte-security_barrier is

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Peter Eisentraut
On sön, 2011-09-25 at 23:49 -0400, Robert Haas wrote: In fact, I've been wondering if we ought to go a step further and not recurse into the sepgsql directory for *any* of the targets. Then we could get rid of the associated configure option, which no longer serves any other purpose, and just

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Kohei KaiGai
2011/9/26 Robert Haas robertmh...@gmail.com: On Sun, Sep 25, 2011 at 10:38 PM, Noah Misch n...@leadboat.com wrote: On Sun, Sep 25, 2011 at 11:22:03AM -0500, Kevin Grittner wrote: Robert Haas  09/25/11 10:58 AM I'm not sure we've been 100% consistent about that, since we previously made

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-26 Thread Peter Eisentraut
On lör, 2011-09-24 at 13:04 -0400, Tom Lane wrote: What if we modified pg_ctl to allow passing configuration parameters through to postmaster, You mean like pg_ctl -o? Note that pg_ctl -o saves the parameters it uses and reapplies them after a restart. So this is not really the way to

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-26 Thread Peter Eisentraut
On lör, 2011-09-24 at 14:02 +0100, Simon Riggs wrote: The semantics are clear: recovery.conf is read first, then postgresql.conf. It's easy to implement (1 line of code) and easy to understand. What's clear about that? My intuition would have been that recovery.conf is read second. -- Sent

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-26 Thread Peter Eisentraut
On sön, 2011-09-25 at 12:58 -0400, Tom Lane wrote: And it's not like we don't break configuration file contents in most releases anyway, so I really fail to see why this one has suddenly become sacrosanct. Well, there is a slight difference. Changes in postgresql.conf parameter names and

[HACKERS] Is there any plan to add unsigned integer types?

2011-09-26 Thread crocket
MySQL already has unsigned INT type, and it has double the range of signed INT type. It's not just the bigger range that UINT type brings. If unsigned INT type exists, I wouldn't have to execute create domain UINT in every database. If INT unsigned and SERIAL unsigned exist, PostgreSQL would

Re: [HACKERS] [v9.2] make_greater_string() does not return a string in some cases

2011-09-26 Thread Peter Eisentraut
On fre, 2011-09-23 at 20:35 +0300, Marcin Mańk wrote: One idea: col like 'foo%' could be translated to col = 'foo' and col = foo || 'zzz' , where 'z' is the largest possible character. This should be good enough for calculating stats. How to find such a character, i do not know. That's

Re: [HACKERS] Is there any plan to add unsigned integer types?

2011-09-26 Thread Peter Eisentraut
On mån, 2011-09-26 at 19:41 +0900, crocket wrote: MySQL already has unsigned INT type, and it has double the range of signed INT type. It's not just the bigger range that UINT type brings. If unsigned INT type exists, I wouldn't have to execute create domain UINT in every database. If INT

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Magnus Hagander
On Mon, Sep 26, 2011 at 06:58, Itagaki Takahiro itagaki.takah...@gmail.com wrote: Hi, I'd like to support UTF-8 text or csv files that has BOM (byte order mark) in COPY FROM command. BOM will be automatically detected and ignored if the file encoding is UTF-8. WIP patch attached. I'm

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Itagaki Takahiro
On Mon, Sep 26, 2011 at 20:12, Magnus Hagander mag...@hagander.net wrote: I like it in general. But if we're looking at the BOM, shouldn't we also look and *reject* the file if it's a BOM for a non-UTF8 file? Say if the BOM claims it's UTF16? -1 because we're depending on manual configuration

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Magnus Hagander
On Mon, Sep 26, 2011 at 13:36, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Mon, Sep 26, 2011 at 20:12, Magnus Hagander mag...@hagander.net wrote: I like it in general. But if we're looking at the BOM, shouldn't we also look and *reject* the file if it's a BOM for a non-UTF8 file? Say

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Andrew Dunstan
On 09/26/2011 07:12 AM, Magnus Hagander wrote: On Mon, Sep 26, 2011 at 06:58, Itagaki Takahiro itagaki.takah...@gmail.com wrote: Hi, I'd like to support UTF-8 text or csv files that has BOM (byte order mark) in COPY FROM command. BOM will be automatically detected and ignored if the file

Re: [HACKERS] Online base backup from the hot-standby

2011-09-26 Thread Fujii Masao
On Fri, Sep 23, 2011 at 12:44 AM, Magnus Hagander mag...@hagander.net wrote: Would it make sense for pg_start_backup() to have the ability to wait for the next restartpoint in a case like this, if we know that FPW has been set? Instead of failing? Or maybe that's just overcomplicating things

Re: [HACKERS] TABLE tab completion

2011-09-26 Thread David Fetter
On Mon, Sep 26, 2011 at 10:37:50AM +0200, Magnus Hagander wrote: On Sun, Sep 25, 2011 at 15:06, Dean Rasheed dean.a.rash...@gmail.com wrote: On 24 September 2011 11:59, Magnus Hagander mag...@hagander.net wrote: TABLE tab completion in psql only completes to tables, not views. but the TABLE

Re: [HACKERS] [v9.2] DROP statement reworks

2011-09-26 Thread Dimitri Fontaine
Dimitri Fontaine dimi...@2ndquadrant.fr writes: The patches are not in git am format nor in patch format, so I could only read them, I didn't install them nor compiled the code, didn't run the regression tests. I've marked the patch as Waiting on Author and referred to my previous message.

Re: [HACKERS] Displaying accumulated autovacuum cost

2011-09-26 Thread Shigeru Hanada
Hi Greg, (2011/08/28 18:54), Greg Smith wrote: Updated patch cleans up two diff mistakes made when backing out the progress report feature. The tip-off I screwed up should have been the absurdly high write rate shown. The usleep was accidentally deleted, so it was running without cost

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 6:28 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2011/9/26 Robert Haas robertmh...@gmail.com: On Sun, Sep 25, 2011 at 10:38 PM, Noah Misch n...@leadboat.com wrote: On Sun, Sep 25, 2011 at 11:22:03AM -0500, Kevin Grittner wrote: Robert Haas  09/25/11 10:58 AM I'm not

[HACKERS] Upgrading Extenions from 8.4

2011-09-26 Thread Dimitri Fontaine
Hi, I'm in the middle of catching-up with pgsql-general and I see several confused users about how to upgrade directly from 8.4. As Tom said, we could easily provide upgrade scripts to handle the move, we just didn't, so there's some more manual work to do. I'm wondering how much work that

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Kohei KaiGai
2011/9/26 Robert Haas robertmh...@gmail.com: On Mon, Sep 26, 2011 at 6:28 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2011/9/26 Robert Haas robertmh...@gmail.com: On Sun, Sep 25, 2011 at 10:38 PM, Noah Misch n...@leadboat.com wrote: On Sun, Sep 25, 2011 at 11:22:03AM -0500, Kevin Grittner

Re: [HACKERS] Inlining comparators as a performance optimisation

2011-09-26 Thread Peter Geoghegan
On 26 September 2011 04:46, Robert Haas robertmh...@gmail.com wrote: I don't want to take time to review this in detail right now, because I don't think it would be fair to put this ahead of things that were submitted for the current CommitFest, but I'm impressed. Thank you. Now that I think

Re: [HACKERS] Is there any plan to add unsigned integer types?

2011-09-26 Thread Merlin Moncure
On Mon, Sep 26, 2011 at 5:41 AM, crocket crockabisc...@gmail.com wrote: MySQL already has unsigned INT type, and it has double the range of signed INT type. It's not just the bigger range that UINT type brings. If unsigned INT type exists, I wouldn't have to execute create domain UINT in

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On sön, 2011-09-25 at 23:49 -0400, Robert Haas wrote: In fact, I've been wondering if we ought to go a step further and not recurse into the sepgsql directory for *any* of the targets. Then we could get rid of the associated configure option, which no

Re: [HACKERS] [v9.2] make_greater_string() does not return a string in some cases

2011-09-26 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On fre, 2011-09-23 at 20:35 +0300, Marcin Mańk wrote: One idea: col like 'foo%' could be translated to col = 'foo' and col = foo || 'zzz' , where 'z' is the largest possible character. This should be good enough for calculating stats. How to find

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 10:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On sön, 2011-09-25 at 23:49 -0400, Robert Haas wrote: In fact, I've been wondering if we ought to go a step further and not recurse into the sepgsql directory for *any* of the targets.

Re: [HACKERS] Is there any plan to add unsigned integer types?

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 10:02 AM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Sep 26, 2011 at 5:41 AM, crocket crockabisc...@gmail.com wrote: MySQL already has unsigned INT type, and it has double the range of signed INT type. It's not just the bigger range that UINT type brings. If

Re: [HACKERS] unaccent contrib

2011-09-26 Thread Oleg Bartunov
On Wed, 21 Sep 2011, Tom Lane wrote: Euler Taveira de Oliveira eu...@timbira.com writes: On 21-09-2011 13:28, Daniel VАzquez wrote: unaccent is compatible with postgresql 8.4 (but not is in their contrib version distribution) No, it is not. AFAICS it is necessary to add some backend code

Re: [HACKERS] Upgrading Extenions from 8.4

2011-09-26 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: I'm in the middle of catching-up with pgsql-general and I see several confused users about how to upgrade directly from 8.4. As Tom said, we could easily provide upgrade scripts to handle the move, we just didn't, so there's some more manual

Re: [HACKERS] [v9.2] make_greater_string() does not return a string in some cases

2011-09-26 Thread Peter Eisentraut
On mån, 2011-09-26 at 10:08 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On fre, 2011-09-23 at 20:35 +0300, Marcin Mańk wrote: One idea: col like 'foo%' could be translated to col = 'foo' and col = foo || 'zzz' , where 'z' is the largest possible character. This should

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 9:51 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: No, you're missing my point completely.  If we use a flexible options syntax here, then we have to decide on what behavior CREATE OR REPLACE should have for all future options, without knowing what they are yet, or what

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Tom Lane
David E. Wheeler da...@kineticode.com cajw2+qdyg1+xlahdqnjs3ackmcsvcdkv_lcapwutwmxl9dz...@mail.gmail.com writes: On Sep 25, 2011, at 9:58 PM, Itagaki Takahiro wrote: I'm thinking about only COPY FROM for reads, but if someone wants to add BOM in COPY TO, we might also support COPY TO WITH BOM

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Tatsuo Ishii
I'd like to support UTF-8 text or csv files that has BOM (byte order mark) in COPY FROM command. BOM will be automatically detected and ignored if the file encoding is UTF-8. WIP patch attached. From RFC3629(http://tools.ietf.org/html/rfc3629#section-6): o A protocol SHOULD forbid use of

Re: [HACKERS] Is there any plan to add unsigned integer types?

2011-09-26 Thread Merlin Moncure
On Mon, Sep 26, 2011 at 9:21 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Sep 26, 2011 at 10:02 AM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Sep 26, 2011 at 5:41 AM, crocket crockabisc...@gmail.com wrote: MySQL already has unsigned INT type, and it has double the range of

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Sep 26, 2011 at 10:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: Another possibility is to remove the Makefile's knowledge of how to run the tests, and change chkselinuxenv into something that both verifies the environment and then launches the

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Tatsuo Ishii
David E. Wheeler da...@kineticode.com cajw2+qdyg1+xlahdqnjs3ackmcsvcdkv_lcapwutwmxl9dz...@mail.gmail.com writes: On Sep 25, 2011, at 9:58 PM, Itagaki Takahiro wrote: I'm thinking about only COPY FROM for reads, but if someone wants to add BOM in COPY TO, we might also support COPY TO WITH

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Kohei KaiGai
2011/9/26 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: On Mon, Sep 26, 2011 at 10:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: Another possibility is to remove the Makefile's knowledge of how to run the tests, and change chkselinuxenv into something that both verifies

Re: [HACKERS] Is there any plan to add unsigned integer types?

2011-09-26 Thread Leonardo Francalanci
  compression is an interesting topic: the guys over at tokudb are making some wild claims...i'm curious if they are real, and what the real tradeoffs are. I don't know how much of the performance they claim comes from compression and how much from the different indexing technique they use

Re: [HACKERS] Adding CORRESPONDING to Set Operations

2011-09-26 Thread Tom Lane
Kerem Kat kerem...@gmail.com writes: In the parser while analyzing SetOperationStmt, larg and rarg needs to be transformed as subqueries. SetOperationStmt can have two fields representing larg and rarg with projected columns according to corresponding: larg_corresponding, rarg_corresponding.

Re: [HACKERS] [PATCH] Use new oom_score_adj without a new compile-time constant

2011-09-26 Thread Dan McGee
On Fri, Sep 23, 2011 at 2:49 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Sep 19, 2011 at 4:36 PM, Dan McGee d...@archlinux.org wrote: [ patch ] I suppose it's Tom who really needs to comment on this, but I'm not too enthusiastic about this approach.  Duplicating the Linux kernel

[HACKERS] random isolation test failures

2011-09-26 Thread Andrew Dunstan
We are seeing numerous occasional buildfarm failures of the fk-deadlock2 isolation test, that look like this: *** *** 32,39 step s2u1: UPDATE B SET Col2 = 1 WHERE BID = 2; step s1u2: UPDATE B SET Col2 = 1 WHERE BID = 2;waiting ... step s2u2: UPDATE B

Re: [HACKERS] [v9.2] make_greater_string() does not return a string in some cases

2011-09-26 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On mån, 2011-09-26 at 10:08 -0400, Tom Lane wrote: No, it's a hundred times worse than that, because in collations other than C there typically *is* no total order. The collation behavior of many characters is context-sensitive, thanks to the

Re: [HACKERS] random isolation test failures

2011-09-26 Thread Kevin Grittner
Andrew Dunstan and...@dunslane.net wrote: We are seeing numerous occasional buildfarm failures of the fk-deadlock2 isolation test If this is harmless, we could provide an alternative results file as a simple fix. If it's not harmless, it should be fixed. I agree, but don't look at me.

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 11:09 AM, Tatsuo Ishii is...@postgresql.org wrote: David E. Wheeler da...@kineticode.com cajw2+qdyg1+xlahdqnjs3ackmcsvcdkv_lcapwutwmxl9dz...@mail.gmail.com writes: On Sep 25, 2011, at 9:58 PM, Itagaki Takahiro wrote: I'm thinking about only COPY FROM for reads, but if

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 11:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Sep 26, 2011 at 10:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: Another possibility is to remove the Makefile's knowledge of how to run the tests, and change chkselinuxenv into

[HACKERS] Multixact truncation for FK locks patch

2011-09-26 Thread Alvaro Herrera
I've been continuing work on modifying the system to let foreign keys coexist concurrently with updates that do not touch the key columns. I've made a lot of progress and things seem to be working rather well. However, I just struck an obstacle that seemed problematic: handling the truncation of

Re: [HACKERS] random isolation test failures

2011-09-26 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: We are seeing numerous occasional buildfarm failures of the fk-deadlock2 isolation test, Yeah, I complained about this already, but Kevin disclaims all responsibility for the fk isolation tests. It looks like Alvaro and Noah Misch are the people to

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 1:15 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Sep 26, 2011 at 11:09 AM, Tatsuo Ishii is...@postgresql.org wrote: Suppose a user uses brain-dead editor, which does not accept UTF-8 without BOM. Maybe this needs to be an

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Florian Pflug
On Sep26, 2011, at 11:50 , Fujii Masao wrote: Currently, if a reference to an invalid page is found during recovery, its information is saved in hash table invalid_page_tab. Then, if such a reference is resolved, its information is removed from the hash table. If there is unresolved

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Sep 26, 2011 at 11:09 AM, Tatsuo Ishii is...@postgresql.org wrote: Suppose a user uses brain-dead editor, which does not accept UTF-8 without BOM. Maybe this needs to be an optional behavior, controlled by some COPY option. I'm not excited

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The thing that makes me doubt that is this comment from Tatsuo Ishii: TI COPY explicitly specifies the encoding (to be UTF-8 in this case). So TI I think we should not regard U+FEFF as BOM in COPY, rather we should TI regard U+FEFF as ZERO WIDTH

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 1:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The thing that makes me doubt that is this comment from Tatsuo Ishii: TI COPY explicitly specifies the encoding (to be UTF-8 in this case).  So TI I think we should not regard U+FEFF

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Sep 26, 2011 at 1:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The thing that makes me doubt that is this comment from Tatsuo Ishii: TI COPY explicitly specifies the encoding (to be UTF-8 in this case).  

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Peter Eisentraut
On tis, 2011-09-27 at 00:09 +0900, Tatsuo Ishii wrote: Suppose a user uses brain-dead editor, which does not accept UTF-8 without BOM. I would first like to see evidence that such an editor exists. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 5:58 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: I think it is.  If you create a view that involves an RTE, the node tree is going to get stored in pg_rewrite.ev_action.  And it's going to include the security_barrier attribute, because you added outfuncs support for

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Peter Eisentraut
On mån, 2011-09-26 at 13:19 -0400, Robert Haas wrote: The thing that makes me doubt that is this comment from Tatsuo Ishii: TI COPY explicitly specifies the encoding (to be UTF-8 in this case). So TI I think we should not regard U+FEFF as BOM in COPY, rather we should TI regard U+FEFF as

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Robert Haas
On Mon, Sep 26, 2011 at 2:38 PM, Peter Eisentraut pete...@gmx.net wrote: On mån, 2011-09-26 at 13:19 -0400, Robert Haas wrote: The thing that makes me doubt that is this comment from Tatsuo Ishii: TI COPY explicitly specifies the encoding (to be UTF-8 in this case). So TI I think we should

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Andrew Dunstan
On 09/26/2011 02:38 PM, Peter Eisentraut wrote: On mån, 2011-09-26 at 13:19 -0400, Robert Haas wrote: The thing that makes me doubt that is this comment from Tatsuo Ishii: TI COPY explicitly specifies the encoding (to be UTF-8 in this case). So TI I think we should not regard U+FEFF as BOM

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Peter Eisentraut
On mån, 2011-09-26 at 14:44 -0400, Robert Haas wrote: We did recently accept a patch for psql -f to skip over a UTF-8 byte-order mark. We had a lot of this same discussion there. But that case is different, because zero-width, non-breaking space has no particular meaning in an SQL script

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Brar Piening
Tom Lane wrote: Putting a BOM into UTF8 data is flat out invalid per spec --- the fact that Microsloth does it does not make it standards-conformant. Could you share a pointer to the spec? All I've ever heard is that a BOM is optional for UTF-8 but not forbidden. The Unicode FAQ

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Brar Piening
Tom Lane wrote: Yeah, that's a reasonable argument for rejecting the patch altogether. I'm not qualified to decide whether it outweighs the we need to be able to read Notepad output argument. Actually it's not only notepad. I quite often find myself doing something like the following when

Re: [HACKERS] psql setenv command

2011-09-26 Thread Josh Kupershmidt
On Thu, Sep 15, 2011 at 7:02 PM, Andrew Dunstan and...@dunslane.net wrote: On Thu, September 15, 2011 6:10 pm, Josh Kupershmidt wrote: [need way to show current values] \! echo $foo (which is how I tested the patch, of course) Ah, right. IMO it'd be helpful to mention that echo example in

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Brar Piening
Robert Haas wrote: The thing that makes me doubt that is this comment from Tatsuo Ishii: TI COPY explicitly specifies the encoding (to be UTF-8 in this case). So TI I think we should not regard U+FEFF as BOM in COPY, rather we should TI regard U+FEFF as ZERO WIDTH NO-BREAK SPACE. If a BOM

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Simon Riggs
On Mon, Sep 26, 2011 at 10:50 AM, Fujii Masao masao.fu...@gmail.com wrote: Currently, if a reference to an invalid page is found during recovery, its information is saved in hash table invalid_page_tab. Then, if such a reference is resolved, its information is removed from the hash table. If

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Kohei KaiGai
2011/9/26 Robert Haas robertmh...@gmail.com: On Mon, Sep 26, 2011 at 5:58 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: I think it is.  If you create a view that involves an RTE, the node tree is going to get stored in pg_rewrite.ev_action.  And it's going to include the security_barrier

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Kohei KaiGai
How about this fix on regression test of sepgsql? It disables to launch regression test together with other modules, and adds its own build target sepgsql-installcheck that launches chkselinuxenv script then pg_regress command as currently we are doing. It allows users to launch regression test

Re: [HACKERS] patch: plpgsql - remove unnecessary ccache search when a array variable is updated

2011-09-26 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2011/9/23 Robert Haas robertmh...@gmail.com: On Fri, Sep 23, 2011 at 12:26 AM, Pavel Stehule pavel.steh...@gmail.com wrote: I fixed crash that described Tom. Do you know about other? No, I just don't see a new version of the patch. sorry - my

Re: [HACKERS] Support UTF-8 files with BOM in COPY FROM

2011-09-26 Thread Tom Lane
Brar Piening b...@gmx.de writes: Citing from the Unicode FAQ again: Q: Where is a BOM useful? A: A BOM is useful at the beginning of files that are typed as text, but for which it is not known whether they are in big or little endian format—it can also serve as a hint indicating that the

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: However, the interface to reference reloptions are designed to pull this information with Relation pointer, rather than lsyscache, so I implemented this revision with a new rte-security_barrier member. This approach will guarantee that we can never

Re: [HACKERS] contrib/sepgsql regression tests are a no-go

2011-09-26 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: How about this fix on regression test of sepgsql? IMO, the fundamental problem with the sepgsql regression tests is that they think they don't need to play by the rules that apply to every other PG regression test. I don't think this patch is fixing that

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: Sorry, are you saying the current (in other words, rte-security_barrier stores the state of reloption) approach is not a good idea? Yes. I think the same as Robert: the way to handle this is to store it in RelOptInfo for the duration of planning, and

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Kohei KaiGai
2011/9/26 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: However, the interface to reference reloptions are designed to pull this information with Relation pointer, rather than lsyscache, so I implemented this revision with a new rte-security_barrier member. This

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Kohei KaiGai
2011/9/26 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: Sorry, are you saying the current (in other words, rte-security_barrier stores the state of reloption) approach is not a good idea? Yes.  I think the same as Robert: the way to handle this is to store it in

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Robert Haas
On Mon, Sep 12, 2011 at 3:31 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: The Part-2 tries to tackles a leaky-view scenarios by functions with very tiny cost estimation value. It was same one we had discussed in the commitfest-1st. It prevents to launch functions earlier than ones come from

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Mon, Sep 26, 2011 at 10:50 AM, Fujii Masao masao.fu...@gmail.com wrote: To prevent the above problem, we should write the contents of the hash table to the disk for every restartpoints, I think. Then, when the server starts recovery, it should

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: One possible idea not to store the flag in RangeTblEntry is to utilize rte-relid to show the relation-id of the source view, when rtekind is RTE_SUBQUERY; that enables to pull the security_barrier flag in executor stage. Maybe I'm confused here, but

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-26 Thread Kohei KaiGai
2011/9/26 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: One possible idea not to store the flag in RangeTblEntry is to utilize rte-relid to show the relation-id of the source view, when rtekind is RTE_SUBQUERY; that enables to pull the security_barrier flag in executor

Re: [HACKERS] psql setenv command

2011-09-26 Thread Jeff Janes
On Mon, Sep 26, 2011 at 12:07 PM, Josh Kupershmidt schmi...@gmail.com wrote: On Thu, Sep 15, 2011 at 7:02 PM, Andrew Dunstan and...@dunslane.net wrote: On Thu, September 15, 2011 6:10 pm, Josh Kupershmidt wrote: [need way to show current values] \! echo $foo (which is how I tested the patch,

[HACKERS] EXECUTE tab completion

2011-09-26 Thread Andreas Karlsson
Hi, Magnus's patch for adding tab completion of views to the TABLE statement reminded me of a minor annoyance of mine -- that EXECUTE always completes with PROCEDURE as if it would have been part of CREATE TRIGGER ... EXECUTE even when it is the first word of the line. Attached is a simple

Re: [HACKERS] psql setenv command

2011-09-26 Thread Andrew Dunstan
On 09/26/2011 05:07 PM, Jeff Janes wrote: But in any case, considering that we are both wondering if it works on Windows, I think that argues that an automatic regression test would be very handy. I think an automated test should be possible. Something like: \setenv PGFOO blurfl \!

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Florian Pflug
On Sep26, 2011, at 22:39 , Tom Lane wrote: It might be worthwhile to invoke XLogCheckInvalidPages() as soon as we (think we) have reached consistency, rather than leaving it to be done only when we exit recovery mode. I believe we also need to prevent the creation of restart points before

Re: [HACKERS] fix for pg_upgrade

2011-09-26 Thread Bruce Momjian
panam wrote: Hi Bruce, on the old DB I've got 465783 as oid whereas on the new one it is 16505. is not in the dump file (old db), even 16385 (i guess this is a typo here) or 16505 are not. The only line in which 465783 could be found is I need to see the lines after this. Is that

Re: [HACKERS] [BUGS] BUG #6218: TRAP: FailedAssertion( !(owner-nsnapshots == 0), File: resowner.c, Line: 365)

2011-09-26 Thread Tom Lane
I wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Shall I work on a fix? I expect you are plenty busy with commitfest stuff, but please let me know otherwise. I have what-I-think-is-the-fix pretty clear in my own mind, so let me give it a try. If it doesn't work I'll bounce it

Re: [HACKERS] [BUGS] BUG #6218: TRAP: FailedAssertion( !(owner-nsnapshots == 0), File: resowner.c, Line: 365)

2011-09-26 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun sep 26 20:59:45 -0300 2011: Well, I soon ran into the issue that delaying the snapshot release makes TopTransactionResourceOwner spit up. After some reflection I decided that the real problem is a circular dependency: snapshot management must be

Re: [HACKERS] [BUGS] BUG #6218: TRAP: FailedAssertion( !(owner-nsnapshots == 0), File: resowner.c, Line: 365)

2011-09-26 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: To be honest, I panicked for a second when I saw the new SnapshotResetXmin call, before I realized that it wasn't necessary before. The serializable case makes more sense the patched way, I think. Yeah, in the old coding, SnapshotResetXmin

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Fujii Masao
On Tue, Sep 27, 2011 at 7:28 AM, Florian Pflug f...@phlo.org wrote: On Sep26, 2011, at 22:39 , Tom Lane wrote: It might be worthwhile to invoke XLogCheckInvalidPages() as soon as we (think we) have reached consistency, rather than leaving it to be done only when we exit recovery mode. I

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-26 Thread Andrew Dunstan
On 09/25/2011 02:39 PM, Joshua Berkus wrote: There might be a use case for a separate directive include_if_exists, or some such name. But I think the user should have to tell us very clearly that it's okay for the file to not be found. Better to go back to include_directory, then. I

Re: [HACKERS] random isolation test failures

2011-09-26 Thread Noah Misch
On Mon, Sep 26, 2011 at 01:10:27PM -0400, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: We are seeing numerous occasional buildfarm failures of the fk-deadlock2 isolation test, Yeah, I complained about this already, but Kevin disclaims all responsibility for the fk isolation

Re: [HACKERS] fix for pg_upgrade

2011-09-26 Thread panam
Hi Bruce, on the old DB I've got 465783 as oid whereas on the new one it is 16505. is not in the dump file (old db), even 16385 (i guess this is a typo here) or 16505 are not. The only line in which 465783 could be found is Is that enough information or should I send the whole dump? That's a

Re: [HACKERS] pg_upgrade automatic testing

2011-09-26 Thread Bruce Momjian
Peter Eisentraut wrote: 8.4 - master upgrade fails like this: Restoring user relation files Mismatch of relation names in database regression: old name pg_toast.pg_toast_27437, new name pg_toast.pg_toast_27309 Failure, exiting This has been 100% reproducible for me. I can now reproduce

Re: [HACKERS] Online base backup from the hot-standby

2011-09-26 Thread Fujii Masao
On Mon, Sep 26, 2011 at 11:39 AM, Steve Singer ssinger...@sympatico.ca wrote: I have looked at both Jun's patch from Sept 13 and Fujii's updates to the patch.  I agree that Fujii's updated version should be used as the basis for changes going forward.   My comments below refer to that version

Re: [HACKERS] random isolation test failures

2011-09-26 Thread Alvaro Herrera
Excerpts from Noah Misch's message of lun sep 26 21:57:40 -0300 2011: These sporadic failures happen whenever the test case takes longer than deadlock_timeout (currently 100ms for these tests) to setup the deadlock. I outlined some mitigating strategies here:

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes: ISTM that writing an invalid-page table to the disk for every restartpoints is better approach. I still say that's uncalled-for overkill. The invalid-page table is not necessary for recovery, it's only a debugging cross-check. You're more likely to

Re: [HACKERS] random isolation test failures

2011-09-26 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: I just tweaked isolationtester so that it collects the error messages and displays them all together at the end of the test. After seeing it run, I didn't like it -- I think I prefer something more local, so that in the only case where we call

  1   2   >