Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Simon Riggs
On Thu, 2007-03-01 at 17:01 +, Heikki Linnakangas wrote: > I ran oprofile on a COPY FROM to get an overview of where the CPU time > is spent. To my amazement, the function at the top of the list was > PageAddItem with 16% of samples. Excellent. I'm slightly worried though since that seems

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2007-03-01 at 17:01 +, Heikki Linnakangas wrote: I ran oprofile on a COPY FROM to get an overview of where the CPU time is spent. To my amazement, the function at the top of the list was PageAddItem with 16% of samples. Excellent. I'm slightly worried though s

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Simon Riggs
On Fri, 2007-03-02 at 10:09 +, Heikki Linnakangas wrote: > Well, there's one big change: your patch to suppress WAL logging on > tables created in the same transaction. OK, just checking thats what you meant. > All the page locking related functions account for ~10% in total, > including

Re: [PATCHES] Numeric patch to add special-case representations for < 8 bytes

2007-03-02 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > Patric Bechtel <[EMAIL PROTECTED]> writes: >> Tom Lane schrieb am 02.03.2007 14:38: >>> Exact decimal fractions are no longer exact when converted to base 2. > >> I think multiplying with base 10 until it's a whole number, then saving >> that exponent with

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > I'm slightly worried though since that seems to have changed from 8.2, > which I oprofiled over Christmas. If you were testing a case with wider rows than Heikki tested, you'd see less impact --- the cost of the old way was O(N^2) in the number of tuples

Re: [PATCHES] Move cursor support for pl/pgsql

2007-03-02 Thread Tom Lane
Magnus Hagander <[EMAIL PROTECTED]> writes: > I took a stab at implementing MOVE support for cursors in PL/pgsql, > because I wanted it for a project. Attached patch *seems* to work. But.. > Given that this is my first venture into touching bison/flex files, I > may be completely off-target in how

Re: [PATCHES] Move cursor support for pl/pgsql

2007-03-02 Thread Magnus Hagander
Tom Lane wrote: > Magnus Hagander <[EMAIL PROTECTED]> writes: >> I took a stab at implementing MOVE support for cursors in PL/pgsql, >> because I wanted it for a project. Attached patch *seems* to work. But.. >> Given that this is my first venture into touching bison/flex files, I >> may be complet

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > "Simon Riggs" <[EMAIL PROTECTED]> writes: >> I'm slightly worried though since that seems to have changed from 8.2, >> which I oprofiled over Christmas. > > If you were testing a case with wider rows than Heikki tested, you'd see > less impact --- the cost

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Simon Riggs
On Fri, 2007-03-02 at 16:25 +, Gregory Stark wrote: > "Tom Lane" <[EMAIL PROTECTED]> writes: > > > "Simon Riggs" <[EMAIL PROTECTED]> writes: > >> I'm slightly worried though since that seems to have changed from 8.2, > >> which I oprofiled over Christmas. > > > > If you were testing a case wit

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Andrew Dunstan
Simon Riggs wrote: IIRC there are issues with delimiter handling when we have lots of columns in the input on COPY FROM, and num of cols on COPY TO. I've not looked at those recently though. What sort of issues? Anything that breaks on this has catastrophic consequences. cheers andrew

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Simon Riggs
On Fri, 2007-03-02 at 11:58 -0500, Andrew Dunstan wrote: > Simon Riggs wrote: > > > > IIRC there are issues with delimiter handling when we have lots of > > columns in the input on COPY FROM, and num of cols on COPY TO. I've not > > looked at those recently though. > > > > > > What sort of iss

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Andrew Dunstan
Simon Riggs wrote: On Fri, 2007-03-02 at 11:58 -0500, Andrew Dunstan wrote: Simon Riggs wrote: IIRC there are issues with delimiter handling when we have lots of columns in the input on COPY FROM, and num of cols on COPY TO. I've not looked at those recently though. What sor

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Simon Riggs
On Fri, 2007-03-02 at 12:09 -0500, Andrew Dunstan wrote: > OK. I'm still curious to know what the issues are with delimiter handling. Rumours only. Feedback from someone else looking to the problem last year. IIRC there was a feeling that if we didn't have to search for delimiters the COPY FROM

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > Feedback from someone else looking to the problem last year. IIRC there > was a feeling that if we didn't have to search for delimiters the COPY > FROM input parsing could be easier. Of what use is the above comment? You have to parse the input into fie

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > What are *you* thinking? Yes, that patch has that line, but > > log_statement and log_min_duration_statement is going to trigger > > log_min_error_statement so you are going to get the statement printed > > twice. > > What's wrong wi

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Simon Riggs
On Fri, 2007-03-02 at 13:24 -0500, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > Feedback from someone else looking to the problem last year. IIRC there > > was a feeling that if we didn't have to search for delimiters the COPY > > FROM input parsing could be easier. > > Of what

Re: [PATCHES] Final version of IDENTITY/GENERATED patch

2007-03-02 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. --- Zo

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Are people going to be happy that log_statement and > log_min_duration_statement output the statement twice? If those are the only cases you're worried about, a far simpler solution is to clear debug_query_string before instead of after emitting those lo

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > "Simon Riggs" <[EMAIL PROTECTED]> writes: >> Feedback from someone else looking to the problem last year. IIRC there >> was a feeling that if we didn't have to search for delimiters the COPY >> FROM input parsing could be easier. > > Of what use is the abo

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes: >> Of what use is the above comment? You have to parse the input into >> fields somehow. > Well those two requirements aren't inconsistent if you're using fixed-width > input text files. Not that I'm enamoured of s

Re: [PATCHES] A little COPY speedup

2007-03-02 Thread Andrew Dunstan
Gregory Stark wrote: The only file formats I ever wanted when I was doing this kind of stuff is tab separated, all the others (comma separated and (egads) *pipe* separated?!) are just disasters. Others of us have to operate in a world where we don't get to choose the format of data we

Re: [PATCHES] Final version of IDENTITY/GENERATED patch

2007-03-02 Thread Zoltan Boszormenyi
Hi! Thanks. However, in the meantime I made some changes so the IDENTITY column only advances its sequence if it fails its CHECK constraints or UNIQUE indexes. I still have some work with expression indexes. Should I post an incremental patch against this version or a full patch when it's ready?

Re: [PATCHES] Final version of IDENTITY/GENERATED patch

2007-03-02 Thread Bruce Momjian
Zoltan Boszormenyi wrote: > Hi! > > Thanks. > > However, in the meantime I made some changes > so the IDENTITY column only advances its sequence > if it fails its CHECK constraints or UNIQUE indexes. > I still have some work with expression indexes. > Should I post an incremental patch against th

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Are people going to be happy that log_statement and > > log_min_duration_statement output the statement twice? > > If those are the only cases you're worried about, a far simpler solution > is to clear debug_query_string before instea

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > I can create a global variable to control this, but the new elog level > seemed cleaner. What I don't like about the proposed patch is that it's nonorthogonal. I see no reason to suppose that LOG is the only possible elevel for which it might be interest

[PATCHES] cosmetic patch to large object regression test

2007-03-02 Thread Jeremy Drake
Since I have now learned that it is possible to input values in hex in postgres, I submit this patch to clean up the ugly workaround I did in the largeobject regression test to try to input hex values. It does not change the functionality of the test at all, it just makes it more readable. In det

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > I can create a global variable to control this, but the new elog level > > seemed cleaner. > > What I don't like about the proposed patch is that it's nonorthogonal. > I see no reason to suppose that LOG is the only possible elevel fo

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Perhaps the best thing would be to define an additional ereport >> auxiliary function, say errprintstmt(bool), that could set a flag >> in the current elog stack entry to control suppression of STATEMENT. >> This would mean you couldn't

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> Perhaps the best thing would be to define an additional ereport > >> auxiliary function, say errprintstmt(bool), that could set a flag > >> in the current elog stack entry to control suppression of STATEMENT. > >>

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> It wouldn't really be any different from errcode(), but if you want >> I'll do it. > If you would just add the infrastructure I can add the LOG part. OK, I applied a patch that covers the same territory as your patch of Wednesday even

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> It wouldn't really be any different from errcode(), but if you want > >> I'll do it. > > > If you would just add the infrastructure I can add the LOG part. > > OK, I applied a patch that covers the same territory

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-03-02 Thread Bruce Momjian
I will rework this before application to use LOG level. Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. ---

Re: [PATCHES] cosmetic patch to large object regression test

2007-03-02 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. --- Je

Re: [PATCHES] POSIX shared memory support

2007-03-02 Thread Chris Marcellino
In case you haven't had enough, here is another version of the code to make Postgres use POSIX shared memory. Along with the issues that have already been addressed, this version ensures that orphaned backends are not in the database when restarting Postgres by using a single 1 byte SysV se

[PATCHES] resetStringInfo

2007-03-02 Thread Neil Conway
Attached is a patch that makes a minor addition to the StringInfo API: resetStringInfo(), which clears the current content of the StringInfo but leaves it valid for future operations. I needed this for an external project, but ISTM this would be worth including in mainline: I'd imagine it's a fair