Re: [GENERAL] Testing 9.0beta3 and pg_upgrade

2010-07-12 Thread Thomas Kellerer
Craig Ringer, 13.07.2010 05:11: On 13/07/10 05:29, Thomas Kellerer wrote: I would suggest to either manually change the autocommit mode from within pg_upgrade or to add a note in the manual to disable/remove this setting from psqlrc.conf before running pg_upgrade. Personally I think the first o

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Joe Conway
On 07/12/2010 11:07 PM, Andrew Bartley wrote: > > I still need some way of finding redundant functions A bit of a blunt instrument, but you could log all statements for a while, and then grep through the logs using a list of all functions of interest to see which ones never show up. Be wary of th

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Andras Fabian
Hi Scott, Although I can't guarantee for 100% that there was no RAID rebuild at some point, I am almost sure that it wasn't the case. Two machines - the ones which were already in production - exhibited this problem. Both of them were already up for some weeks. Now, the reboot rather "fixed" on

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Andrew Bartley
Thanks Alexander, Wish i had thought of that. I still need some way of finding redundant functions Thanks again Andrew On 13 July 2010 15:38, Alexander Pyhalov wrote: > Hello. > When we moved old projects from postgresql 7.x to 8.4, I just looked at > modification time for files in base// .

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Alexander Pyhalov
Hello. When we moved old projects from postgresql 7.x to 8.4, I just looked at modification time for files in base// . So, I could determine, that some databases were inactive (precisely,not updated) for about a year and move them to archive... Andrew Bartley wrote: Is there an effective way

Re: [GENERAL] Why can't I see the definition of my relations

2010-07-12 Thread A. Kretschmer
In response to Andrew Falanga : > Hi, > > I just finished defining a couple of tables with PgAdmin III and I'm > seeing something peculiar. I'm not sure what the problem is. When I > connect to the DB using psql and do "\d " I get an error saying > that there's not relations by that name. What?

[GENERAL] Why can't I see the definition of my relations

2010-07-12 Thread Andrew Falanga
Hi, I just finished defining a couple of tables with PgAdmin III and I'm seeing something peculiar. I'm not sure what the problem is. When I connect to the DB using psql and do "\d " I get an error saying that there's not relations by that name. What? When I do, "\d" I see the tables listed.

Re: [GENERAL] No PL/PHP ? Any reason?

2010-07-12 Thread Alvaro Herrera
Excerpts from Joshua D. Drake's message of mar jul 13 00:00:07 -0400 2010: > On Tue, 2010-07-13 at 03:42 +, Greg Sabino Mullane wrote: > > -BEGIN PGP SIGNED MESSAGE- > > Hash: RIPEMD160 > > > > > > Joshua D. Drake wrote: > > >> * No trusted/untrusted versions > > > > > > This is false

Re: [GENERAL] \COPY ... CSV with hex escapes

2010-07-12 Thread Craig Ringer
On 08/07/10 17:42, Alban Hertroys wrote: > On 8 Jul 2010, at 4:21, Craig Ringer wrote: > >> Yes, that's ancient. It is handled quite happily by \copy in csv mode, >> except that when csv mode is active, \xnn escapes do not seem to be >> processed. So I can have *either* \xnn escape processing *or*

Re: [GENERAL] No PL/PHP ? Any reason?

2010-07-12 Thread Joshua D. Drake
On Tue, 2010-07-13 at 03:42 +, Greg Sabino Mullane wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: RIPEMD160 > > > Joshua D. Drake wrote: > >> * No trusted/untrusted versions > > > > This is false. There are both. > > Ah, good news, glad I was misinformed. I'm curious, what > mechanism

Re: [GENERAL] No PL/PHP ? Any reason?

2010-07-12 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Joshua D. Drake wrote: >> * No trusted/untrusted versions > > This is false. There are both. Ah, good news, glad I was misinformed. I'm curious, what mechanism does it use for trusted? >> * Not even in contrib or pgfoundry or github > No. No

Re: [GENERAL] Testing 9.0beta3 and pg_upgrade

2010-07-12 Thread Craig Ringer
On 13/07/10 05:29, Thomas Kellerer wrote: > I would suggest to either manually change the autocommit mode from > within pg_upgrade or to add a note in the manual to disable/remove this > setting from psqlrc.conf before running pg_upgrade. Personally I think > the first option would be the better o

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Craig Ringer
On 12/07/10 21:03, Andras Fabian wrote: > This STDOU issue gets even weirder. Now I have set up our two new servers > (identical hw/sw) as I would have needed to do so anyways. After having PG > running, I also set up the same test scenario as I have it on our problematic > servers, and started

Re: [GENERAL] PostgreSQL 9.0 beta 3 release announcement

2010-07-12 Thread Bruce Momjian
Thomas Kellerer wrote: > Bruce Momjian wrote on 12.07.2010 21:34: > > Thom Brown wrote: > >> Could someone clarify the info in this paragraph: > >> > >> "Note that, due to a system catalog change, an initdb and database > >> reload will be required for upgrading from 9.0Beta1. We encourage > >> use

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Scott Marlowe
On Mon, Jul 12, 2010 at 7:03 AM, Andras Fabian wrote: > This STDOU issue gets even weirder. Now I have set up our two new servers > (identical hw/sw) as I would have needed to do so anyways. After having PG > running, I also set up the same test scenario as I have it on our problematic > server

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Andrew Bartley
Thanks Greg, It seems that the underlying stats tables are reset on a periodic basis, can i stop this process? Is it a .conf setting? I have had a good look around, nothing sticks out. If I can stop it, then i could use pg_stat_reset() then monitor the stat views over an extended period without t

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Greg Smith
Andrew Bartley wrote: Unfortunately these views only give me what appears to be a certain time frame. This does not help all that much. It will give a list of tables, indexes and sequences that have been used in the time frame, so that is at least a start. You can use pg_stat_reset() to set

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Andrew Bartley
Thanks Joe, Unfortunately these views only give me what appears to be a certain time frame. This does not help all that much. It will give a list of tables, indexes and sequences that have been used in the time frame, so that is at least a start. It would be good if there was a timestamp (last

Re: [GENERAL] Redundant database objects.

2010-07-12 Thread Joe Conway
On 07/12/2010 02:40 PM, Andrew Bartley wrote: > We have a large number of orphaned or redundant tables, views, and > functions, due to many years of inadequate source management. > > We are running " PostgreSQL 8.3.7 on i486-pc-linux-gnu, compiled by GCC > gcc-4.3.real (Debian 4.3.2-1.1) 4.3.2 "

[GENERAL] Redundant database objects.

2010-07-12 Thread Andrew Bartley
Hi all, Our project has been running for 10 years now. We have a large number of orphaned or redundant tables, views, and functions, due to many years of inadequate source management. We are running " PostgreSQL 8.3.7 on i486-pc-linux-gnu, compiled by GCC gcc-4.3.real (Debian 4.3.2-1.1) 4.3.2 "

[GENERAL] Testing 9.0beta3 and pg_upgrade

2010-07-12 Thread Thomas Kellerer
Hi, I'm trying pg_upgrade on my Windows installation and I have two suggestions for the manual regarding pg_upgrade: When specifying directories, pg_upgrade *requires* a forward slash as the path separator. This is (still) uncommon in the Windows world (although Windows does support it) and

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Scott Marlowe
Please don't top post. On Mon, Jul 12, 2010 at 2:20 PM, Tom Wilcox wrote: > On 12/07/2010 19:26, Scott Marlowe wrote: >> >> On Mon, Jul 12, 2010 at 7:57 AM, Thom Brown  wrote: >> >>> >>> On 12 July 2010 14:50, Tom Wilcox  wrote: >>> Hi Thom, I am performing update statements t

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Tom Wilcox
I could perform the settings manually (set config, restart svr, execute script, come back 2 days later, reset config, restart svr, execute more script,...), but that sort of defeats the point. My aim to have the simplest, automatic setup possible. Preferably completely contained within PostgreS

Re: [GENERAL] PostgreSQL 9.0 beta 3 release announcement

2010-07-12 Thread Thomas Kellerer
Bruce Momjian wrote on 12.07.2010 21:34: Thom Brown wrote: Could someone clarify the info in this paragraph: "Note that, due to a system catalog change, an initdb and database reload will be required for upgrading from 9.0Beta1. We encourage users to use this opportunity to test pg_upgrade for

Re: [GENERAL] PostgreSQL 9.0 beta 3 release announcement

2010-07-12 Thread Bruce Momjian
Thom Brown wrote: > Could someone clarify the info in this paragraph: > > "Note that, due to a system catalog change, an initdb and database > reload will be required for upgrading from 9.0Beta1. We encourage > users to use this opportunity to test pg_upgrade for the upgrade from > Beta2 or an ear

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Greg Smith
Andres Freund wrote: What you can change (and that makes quite a bit of sense in some situations) is the "synchronous_commit" setting. Right. In almost every case where people think they want to disable fsync, what they really should be doing instead is turning off synchronous commit--whi

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Greg Smith
Andras Fabian wrote: - all fast servers show the COPY process as being in the state Rs ("runnable (on run queue)") - on the still slow server, this process is in 9 out of 10 samples in Ds ("uninterruptible sleep (usually IO)") I've run into significant performance regressions in PostgreSQL

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Scott Marlowe
On Mon, Jul 12, 2010 at 7:57 AM, Thom Brown wrote: > On 12 July 2010 14:50, Tom Wilcox wrote: >> Hi Thom, >> >> I am performing update statements that are applied to a single table that is >> about 96GB in size. These updates are grouped together in a single >> transaction. This transaction runs

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Brad Nicholson
On Mon, 2010-07-12 at 14:57 +0100, Thom Brown wrote: > On 12 July 2010 14:50, Tom Wilcox wrote: > > Hi Thom, > > > > I am performing update statements that are applied to a single table that is > > about 96GB in size. These updates are grouped together in a single > > transaction. This transaction

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Pavel Stehule
2010/7/12 Josip Rodin : > On Mon, Jul 12, 2010 at 04:38:48PM +0200, Pavel Stehule wrote: >> 2010/7/12 Josip Rodin : >> > On Mon, Jul 12, 2010 at 02:06:43PM +0800, Craig Ringer wrote: >> >> Meh, personally I'll stick to the good old profiling methods "is it fast >> >> enough", "\timing", and "explai

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Josip Rodin
On Mon, Jul 12, 2010 at 04:38:48PM +0200, Pavel Stehule wrote: > 2010/7/12 Josip Rodin : > > On Mon, Jul 12, 2010 at 02:06:43PM +0800, Craig Ringer wrote: > >> Meh, personally I'll stick to the good old profiling methods "is it fast > >> enough", "\timing", and "explain analyze". > > > > I agree. S

Re: [GENERAL] ERROR: canceling statement due to statement timeout

2010-07-12 Thread Tim
On 6 July 2010 16:36, Vick Khera wrote: > On Fri, Jul 2, 2010 at 12:22 PM, Tim wrote: >> I've had a website up for a couple of months and it's starting to get >> these db timeouts as traffic has increased to say 1k pageviews a day. >> > > Are you using any two-phase commit (ie, prepared transacti

Re: [GENERAL] getting the last N tuples of a query

2010-07-12 Thread Merlin Moncure
On Thu, Jul 8, 2010 at 9:09 PM, Kenichiro Tanaka wrote: > Hello. > > I agree Ben. > But,I try your question as an SQL puzzle. > Doses this SQL meet what you want? > > select * from wantlast offset (select count(*)-10 from wantlast); that works, but for any non trivial query it's not optimal becau

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Andres Freund
On Monday 12 July 2010 15:29:14 Tom Wilcox wrote: > Hi, > > Is it possible to configure postgres from SQL? > > I am interested in turning off fsync for a set of queries (that take > ages to run) and then turn fsync back on again afterwards. disabling fsync is nearly never a good idea. What you c

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Guillaume Lelarge
Le 12/07/2010 17:02, Tom Wilcox a écrit : > Hi Thom, > > Yeah They can be divided up, but my main issue is that I would like > these functions wrapped up so that the client (who has little to no > experience using PostgreSQL) can just run a SQL function that will > execute all of these updates and

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Tom Wilcox
Hi Thom, Yeah They can be divided up, but my main issue is that I would like these functions wrapped up so that the client (who has little to no experience using PostgreSQL) can just run a SQL function that will execute all of these updates and prepare many tables and functions for a product.

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Pavel Stehule
2010/7/12 Josip Rodin : > On Mon, Jul 12, 2010 at 02:06:43PM +0800, Craig Ringer wrote: >> Meh, personally I'll stick to the good old profiling methods "is it fast >> enough", "\timing", and "explain analyze". > > I agree. Some hint could be included in 'explain analyze' output, maybe just > to sep

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Josip Rodin
On Mon, Jul 12, 2010 at 02:06:43PM +0800, Craig Ringer wrote: > Meh, personally I'll stick to the good old profiling methods "is it fast > enough", "\timing", and "explain analyze". I agree. Some hint could be included in 'explain analyze' output, maybe just to separate the timings for things th

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Adrian Klaver
On Monday 12 July 2010 6:29:14 am Tom Wilcox wrote: > Hi, > > Is it possible to configure postgres from SQL? Yes to a degree, see here: http://www.postgresql.org/docs/8.4/interactive/functions-admin.html > > I am interested in turning off fsync for a set of queries (that take > ages to run) and t

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Thom Brown
On 12 July 2010 14:50, Tom Wilcox wrote: > Hi Thom, > > I am performing update statements that are applied to a single table that is > about 96GB in size. These updates are grouped together in a single > transaction. This transaction runs until the machine runs out of disk space. > > What I am try

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread tv
> Hi, > > Is it possible to configure postgres from SQL? > > I am interested in turning off fsync for a set of queries (that take > ages to run) and then turn fsync back on again afterwards. There are things that can be changed at runtime using SQL - in that case you may just type "SET enable_seqs

Re: [GENERAL] Configure Postgres From SQL

2010-07-12 Thread Thom Brown
On 12 July 2010 14:29, Tom Wilcox wrote: > Hi, > > Is it possible to configure postgres from SQL? > > I am interested in turning off fsync for a set of queries (that take ages to > run) and then turn fsync back on again afterwards. > > Cheers, > Tom > You can only change that option in postgresql

[GENERAL] Configure Postgres From SQL

2010-07-12 Thread Tom Wilcox
Hi, Is it possible to configure postgres from SQL? I am interested in turning off fsync for a set of queries (that take ages to run) and then turn fsync back on again afterwards. Cheers, Tom -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscri

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Andras Fabian
This STDOU issue gets even weirder. Now I have set up our two new servers (identical hw/sw) as I would have needed to do so anyways. After having PG running, I also set up the same test scenario as I have it on our problematic servers, and started the COPY-to-STDOUT experiment. And you know what

[GENERAL] PostgreSQL 9.0 beta 3 release announcement

2010-07-12 Thread Thom Brown
Could someone clarify the info in this paragraph: "Note that, due to a system catalog change, an initdb and database reload will be required for upgrading from 9.0Beta1. We encourage users to use this opportunity to test pg_upgrade for the upgrade from Beta2 or an earlier version of 9.0. Please re

[GENERAL] PostgreSQL PITR - more doubts

2010-07-12 Thread Jayadevan M
Hello all, One doubt about how PostgreSQL PITR works. Let us say I have all the archived WALs for the past week with archive_command = 'cp -i %p /home/postgres/archive/%f http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Queries about PostgreSQL PITR

2010-07-12 Thread Fujii Masao
On Mon, Jul 12, 2010 at 5:29 PM, Jayadevan M wrote: > Hi, >>Because you didn't disable recovery_target_inclusive, I guess. >> > http://www.postgresql.org/docs/8.4/static/continuous-archiving.html#RECOVERY-TARGET-INCLUSIVE > Thanks. I was almost sure this will fix it. But the issue seems to be > so

Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

2010-07-12 Thread Andras Fabian
Hi Tom (or others), are there some recommended settings/ways to use oprofile on a situation like this??? I got it working, have seen a first profile report, but then managed to completely freeze the server on a second try with different oprofile settings (next tests will go against the newly in

Re: [GENERAL] Queries about PostgreSQL PITR

2010-07-12 Thread Jayadevan M
Hi, >Because you didn't disable recovery_target_inclusive, I guess. > http://www.postgresql.org/docs/8.4/static/continuous-archiving.html#RECOVERY-TARGET-INCLUSIVE Thanks. I was almost sure this will fix it. But the issue seems to be something else. Even if I give a time that is a few more minutes

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Dave Page
On Mon, Jul 12, 2010 at 7:06 AM, Craig Ringer wrote: > It seems like a profiler, which is designed to filter and organize the > collected data, and which can be attached only to specific functions that > you want to know about, might be a better job. As there's already a PL/PgSQL > debugger, some

Re: [GENERAL] simple functions, huge overhead, no cache

2010-07-12 Thread Pavel Stehule
2010/7/12 Craig Ringer : > On 11/07/2010 5:46 AM, Pavel Stehule wrote: > >> any using a non simple expression is very slow - so there can be some >> a warning when people use it. Sometimes people don't know (me too), >> when use expensive expression >> >> for example >> >> rowvar := (10,20) >> >> i