[GENERAL] pg_dump slow with bytea data

2011-03-02 Thread chris r.
Dear list, As discussed extensively in the past [1], pg_dump tends to be slow for tables that contain bytea columns with large contents. Starting with postgres version 8.5 the COPY format of bytea was changed from escape to hex [1], giving ~50% performance boost. However, we experience heavy

Re: [GENERAL] pg_dump slow with bytea data

2011-03-02 Thread chris r.
As discussed extensively in the past [1] Argh, forgot to add the reference: [1]: http://archives.postgresql.org/pgsql-hackers/2009-05/msg00192.php Chris -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

Re: [GENERAL] Dynamic binding in plpgsql function

2011-03-02 Thread Merlin Moncure
On Tue, Mar 1, 2011 at 5:17 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello 2011/3/2 Pierre Racine pierre.rac...@sbf.ulaval.ca: Hi, I would like to write a generic plpgsql function with a text parameter being a callback function name so that my general function can call this

Re: [GENERAL] Dynamic binding in plpgsql function

2011-03-02 Thread Dmitriy Igrishin
2011/3/2 Merlin Moncure mmonc...@gmail.com On Tue, Mar 1, 2011 at 5:17 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello 2011/3/2 Pierre Racine pierre.rac...@sbf.ulaval.ca: Hi, I would like to write a generic plpgsql function with a text parameter being a callback function

Re: [GENERAL] pg_dump slow with bytea data

2011-03-02 Thread Merlin Moncure
On Wed, Mar 2, 2011 at 2:35 AM, chris r. chri...@gmx.net wrote: Dear list, As discussed extensively in the past [1], pg_dump tends to be slow for tables that contain bytea columns with large contents. Starting with postgres version 8.5 the COPY format of bytea was changed from escape to hex

Re: [GENERAL] Dynamic binding in plpgsql function

2011-03-02 Thread Merlin Moncure
On Wed, Mar 2, 2011 at 8:34 AM, Dmitriy Igrishin dmit...@gmail.com wrote: 2011/3/2 Merlin Moncure mmonc...@gmail.com On Tue, Mar 1, 2011 at 5:17 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello 2011/3/2 Pierre Racine pierre.rac...@sbf.ulaval.ca: Hi, I would like to write a

Re: [GENERAL] Dynamic binding in plpgsql function

2011-03-02 Thread Dmitriy Igrishin
2011/3/2 Merlin Moncure mmonc...@gmail.com On Wed, Mar 2, 2011 at 8:34 AM, Dmitriy Igrishin dmit...@gmail.com wrote: 2011/3/2 Merlin Moncure mmonc...@gmail.com On Tue, Mar 1, 2011 at 5:17 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello 2011/3/2 Pierre Racine

Re: [GENERAL] Dynamic binding in plpgsql function

2011-03-02 Thread Pierre Racine
Is EXECUTE slower than a direct assignment call? -Original Message- From: Vibhor Kumar [mailto:vibhor.ku...@enterprisedb.com] Sent: 1 mars 2011 18:24 To: Pierre Racine Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Dynamic binding in plpgsql function On Mar 2, 2011, at 4:31 AM,

Re: [GENERAL] Dynamic binding in plpgsql function

2011-03-02 Thread Merlin Moncure
On Wed, Mar 2, 2011 at 8:54 AM, Dmitriy Igrishin dmit...@gmail.com wrote: 2011/3/2 Merlin Moncure mmonc...@gmail.com Lane...http://postgresql.1045698.n5.nabble.com/improvise-callbacks-in-plpgsql-td2052928.html (for whom nary a day goes by that I am not thankful for his tireless efforts).

Re: [GENERAL] Dynamic binding in plpgsql function

2011-03-02 Thread Merlin Moncure
On Wed, Mar 2, 2011 at 9:06 AM, Pierre Racine pierre.rac...@sbf.ulaval.ca wrote: Is EXECUTE slower than a direct assignment call? It is going to be slower, but how much slower and if it justifies the mechanism is going to be a matter of your requirements, definition of 'slow', and willingness to

[GENERAL] I need your help to get opinions about this situation

2011-03-02 Thread Rayner Julio Rodríguez Pimentel
Hello to everybody, I have this situation that I would to know your opinions about it, to confirm some elements that secure me of use the amazing database system PostgreSQL. I have a database of 1000 tables, 300 of theirs are of major growing with 1 rows daily, the estimate growing for this

[GENERAL] Per-session memory footprint (9.0/windows)

2011-03-02 Thread Hannes Erven
Folks, I run a PG (currently 8.4, but will shortly migrate to 9.0) database on Windows Server 2003 that supports a desktop application which opens a few long-running sessions per user. This is due to the Hibernate persistence layer and the one session per view pattern that is recommended for

Re: [GENERAL] Per-session memory footprint (9.0/windows)

2011-03-02 Thread Scott Mead
On Wed, Mar 2, 2011 at 11:07 AM, Hannes Erven han...@erven.at wrote: Folks, I run a PG (currently 8.4, but will shortly migrate to 9.0) database on Windows Server 2003 that supports a desktop application which opens a few long-running sessions per user. This is due to the Hibernate

Re: [GENERAL] pg_dump slow with bytea data

2011-03-02 Thread Alban Hertroys
On 2 Mar 2011, at 9:35, chris r. wrote: GB). Note that we ran VACUUM FULL on the tables affected. Did you also REINDEX them? Alban Hertroys -- Screwing up is an excellent way to attach something to the ceiling. !DSPAM:737,4d6e8542235882633876383! -- Sent via pgsql-general mailing list

[GENERAL] Index question

2011-03-02 Thread Michael Black
Ok. I have been working with databases a few years but my first real venture in to PostgreSql. I just want a plain simple index regardless if there are duplicates or not. How do I accomplish this in PostgreSql? Michael

Re: *****SPAM***** [GENERAL] Index question

2011-03-02 Thread Scott Ribe
On Mar 2, 2011, at 11:31 AM, Michael Black wrote: Ok. I have been working with databases a few years but my first real venture in to PostgreSql. I just want a plain simple index regardless if there are duplicates or not. How do I accomplish this in PostgreSql? Same as any other SQL

Re: [GENERAL] Index question

2011-03-02 Thread Andrew Sullivan
On Wed, Mar 02, 2011 at 06:31:57PM +, Michael Black wrote: Ok. I have been working with databases a few years but my first real venture in to PostgreSql. I just want a plain simple index regardless if there are duplicates or not. How do I accomplish this in PostgreSql? CREATE

Re: [GENERAL] Index question

2011-03-02 Thread Scott Ribe
On Mar 2, 2011, at 11:43 AM, Michael Black wrote: Thanks Scott. I just did not see the options in the PGAdmin III nor in the doc at You may want to bookmark this: http://www.postgresql.org/docs/9.0/static/sql-commands.html -- Scott Ribe scott_r...@elevated-dev.com

Re: [GENERAL] Index question

2011-03-02 Thread Joshua D. Drake
On Wed, 2011-03-02 at 13:45 -0500, Andrew Sullivan wrote: On Wed, Mar 02, 2011 at 06:31:57PM +, Michael Black wrote: Ok. I have been working with databases a few years but my first real venture in to PostgreSql. I just want a plain simple index regardless if there are duplicates

[GENERAL] Grant question

2011-03-02 Thread Michael Black
Ok. What am I missing here? B_USER is a defined Group Role CREATE ROLE B_USER NOSUPERUSER NOINHERIT NOCREATEDB NOCREATEROLE; GRANT SELECT PRIVILEGES ON b.config_itm TO ROLE B_USER; Nets this --- ERROR: syntax error at or near B_USER LINE

Re: [GENERAL] Index question

2011-03-02 Thread Michael Black
Thank you for the links. Subject: Re: [GENERAL] Index question From: j...@commandprompt.com To: a...@crankycanuck.ca CC: pgsql-general@postgresql.org Date: Wed, 2 Mar 2011 11:05:58 -0800 On Wed, 2011-03-02 at 13:45 -0500, Andrew Sullivan wrote: On Wed, Mar 02, 2011 at 06:31:57PM +,

Re: [GENERAL] Grant question

2011-03-02 Thread Bill Moran
In response to Michael Black michaelblack75...@hotmail.com: Ok. What am I missing here? B_USER is a defined Group Role CREATE ROLE B_USER NOSUPERUSER NOINHERIT NOCREATEDB NOCREATEROLE; GRANT SELECT PRIVILEGES ON b.config_itm TO ROLE B_USER; Nets this

Re: [GENERAL] Grant question

2011-03-02 Thread S G
On Wed, Mar 2, 2011 at 2:30 PM, Michael Black michaelblack75...@hotmail.com wrote: Ok.  What am I missing here?  B_USER is a defined Group Role CREATE ROLE B_USER   NOSUPERUSER NOINHERIT NOCREATEDB NOCREATEROLE; GRANT SELECT PRIVILEGES ON b.config_itm TO ROLE B_USER; Nets this

Re: [GENERAL] Grant question

2011-03-02 Thread Bosco Rama
Michael Black wrote: Ok. What am I missing here? B_USER is a defined Group Role CREATE ROLE B_USER You used double-quotes here. This will preserve case and any non-standard identifier characters (spaces, punctuation, etc.) TO ROLE B_USER; And, thus, you need to use them here. Without

[GENERAL] How to approach dynamic status reporting

2011-03-02 Thread James B. Byrne
I have a situation whereby edi unit record files from an external system are read, parsed and loaded into a PostgreSQL database. As transmissions relating to each transaction are read a log table entry is made by transaction for each type of transmission encountered. The nature of the external

[GENERAL] select DISTINCT not ordering the returned rows

2011-03-02 Thread Ioana Danes
Hi Everyone, I would like to ask for your help finding a temporary solution for my problem. I upgraded postgres from 8.3 to 9.0.3 and I have an issue with the order of the returned rows. The following script is a simplification of my real case: create table tmp_1 (field1 integer, field2

Re: [GENERAL] select DISTINCT not ordering the returned rows

2011-03-02 Thread Ioana Danes
I found it: disabling enable_hashagg --- On Wed, 3/2/11, Ioana Danes ioanasoftw...@yahoo.ca wrote: From: Ioana Danes ioanasoftw...@yahoo.ca Subject: [GENERAL] select DISTINCT not ordering the returned rows To: PostgreSQL General pgsql-general@postgresql.org Received: Wednesday, March 2,

Re: [GENERAL] select DISTINCT not ordering the returned rows

2011-03-02 Thread Tom Lane
Ioana Danes ioanasoftw...@yahoo.ca writes: I upgraded postgres from 8.3 to 9.0.3 and I have an issue with the order of the returned rows. Your application is simply broken if it assumes that DISTINCT results in ordering the rows. The only thing that guarantees that is an ORDER BY. You could

Re: [GENERAL] select DISTINCT not ordering the returned rows

2011-03-02 Thread Thomas Kellerer
Ioana Danes, 02.03.2011 21:35: Hi Everyone, I would like to ask for your help finding a temporary solution for my problem. I upgraded postgres from 8.3 to 9.0.3 and I have an issue with the order of the returned rows. The database is free to return rows in any order it thinks is most

Re: [GENERAL] select DISTINCT not ordering the returned rows

2011-03-02 Thread Ioana Danes
I totally agree with you and the problem is gonna be fixed. I just needed a temporary solution until the patch goes out. Thank you, Ioana --- On Wed, 3/2/11, Tom Lane t...@sss.pgh.pa.us wrote: From: Tom Lane t...@sss.pgh.pa.us Subject: Re: [GENERAL] select DISTINCT not ordering the returned

Re: [GENERAL] Per-session memory footprint (9.0/windows)

2011-03-02 Thread Hannes Erven
Scott, It seems that each of the server postmaster.exe processes takes up approx. 5 MB of server memory (the virtual memory size column in task manager), and I guess this truly is the private memory these processes require. This number is roughly the same for 8.4 and 9.0 .

Re: [GENERAL] Per-session memory footprint (9.0/windows)

2011-03-02 Thread Tom Lane
Hannes Erven han...@erven.at writes: It is still about 5 MB of private memory per idle backend process. Is there anything I can do to optimize? That sounds about the right ballpark for a working backend process with caches loaded up. If that's too much for you, you ought to be using connection

Re: [GENERAL] how to avoid repeating expensive computation in select

2011-03-02 Thread Merlin Moncure
On Tue, Mar 1, 2011 at 2:51 AM, Orhan Kavrakoglu or...@tart.com.tr wrote: I would like to know if there is a way in PostgreSQL to avoid repeating an expensive computation in a SELECT where the result is needed both as a returned value and as an expression in the WHERE clause. I think I've

Re: [GENERAL] PG on two nodes with shared disk ocfs2 drbd

2011-03-02 Thread Jasmin Dizdarevic
hehe... andrew, I appriciate pg and it's free open source features - maybe I've chosen a wrong formulation. in my eyes such a feature is getting more important nowadays. Postgresql-R and -XC are interesting ideas. thanks everybody for the comments regards, jasmin 2011/2/28 Andrew Sullivan

Re: [GENERAL] PG on two nodes with shared disk ocfs2 drbd

2011-03-02 Thread Craig Ringer
On 03/03/11 09:01, Jasmin Dizdarevic wrote: hehe... andrew, I appriciate pg and it's free open source features - maybe I've chosen a wrong formulation. in my eyes such a feature is getting more important nowadays. Why? Shared disk means shared point of failure, and poor redundancy against a

Re: [GENERAL] ERROR: invalid byte sequence for encoding UTF8: 0xc35c

2011-03-02 Thread Jasmin Dizdarevic
@ALL: Isn't it possible and wise to include an (optional) encoder in pgsql? we're importing a lot of data from textfiles, which are not utf-8. we always have to change the encoding in another tool before using COPY. 2011/2/28 Craig Ringer cr...@postnewspapers.com.au On 27/02/11 20:47, AI

Re: [GENERAL] ERROR: invalid byte sequence for encoding UTF8: 0xc35c

2011-03-02 Thread Craig Ringer
On 03/03/11 09:18, Jasmin Dizdarevic wrote: @ALL: Isn't it possible and wise to include an (optional) encoder in pgsql? we're importing a lot of data from textfiles, which are not utf-8. we always have to change the encoding in another tool before using COPY. SET client_encoding -- Craig

[GENERAL] data type

2011-03-02 Thread Nick Raj
Hi, I am writing some function in postgres pl/sql. My function is of type St_ABC((select obj_geom from XYZ),(select boundary_geom from boundary)) I have table XYZ with 20,000 tuples and in boundary, i have only one geometry. In postgres, ST_intersects(obj_geom, boundary_geom) checks each

[GENERAL] close connection more expensive that open connection?

2011-03-02 Thread Rob Sargent
A developer here accidentally flooded a server with connection opens and closes, essentially one per transaction during a multi-threaded data migration process. We were curious if this suggests that connection clean up is more expensive than creation thereby exhausting resources, or if perhaps