Re: [HACKERS] is_array_type vs type_is_array

2007-06-07 Thread Jeremy Drake
On Thu, 7 Jun 2007, Tom Lane wrote: Jeremy Drake [EMAIL PROTECTED] writes: Was there some change in functionality reason for renaming is_array_type to type_is_array? Just to sync style with type_is_enum ... there were more of the latter than the former. OK, so it is safe to just #define

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Heikki Linnakangas
Greg Smith wrote: On Wed, 6 Jun 2007, Heikki Linnakangas wrote: The original patch uses bgwriter_all_max_pages to set the minimum rate. I think we should have a separate variable, checkpoint_write_min_rate, in KB/s, instead. Completely agreed. There shouldn't be any coupling with the

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Hannu Krosing
Ühel kenal päeval, K, 2007-06-06 kell 11:03, kirjutas Tom Lane: Heikki Linnakangas [EMAIL PROTECTED] writes: GUC summary and suggested default values checkpoint_write_percent = 50 # % of checkpoint interval to spread out writes

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-07 Thread Heikki Linnakangas
Florian G. Pflug wrote: Jeff Davis wrote: Are you referring to the size of the xip array being a problem? Would it help to tie the size of the xip array to max_connections? I understand that max_connections might be greater on the master, but maybe something similar? Thats what I currently

Re: [HACKERS] How do we create the releases?

2007-06-07 Thread Magnus Hagander
On Wed, Jun 06, 2007 at 07:56:25PM -0300, Marc G. Fournier wrote: - --On Tuesday, June 05, 2007 10:28:58 +0300 Devrim GÜNDÜZ [EMAIL PROTECTED] wrote: Hi Marc, Is there a written procedure about creating tarballs? I'd like to start working on 8.3 RPMs and I want to know what I should

Re: [HACKERS] How do we create the releases?

2007-06-07 Thread Peter Eisentraut
Magnus Hagander wrote: Is this script in CVS somewhere? I know it's not in the main backend repo. Most of what these scripts have done in the past has been systematized and folded into the make dist target in the mean time, and AFAICT, we could integrate the documentation preparation as well,

Re: [HACKERS] How do we create the releases?

2007-06-07 Thread Magnus Hagander
On Thu, Jun 07, 2007 at 01:12:06PM +0200, Peter Eisentraut wrote: Magnus Hagander wrote: Is this script in CVS somewhere? I know it's not in the main backend repo. Most of what these scripts have done in the past has been systematized and folded into the make dist target in the mean

Re: [HACKERS] How do we create the releases?

2007-06-07 Thread Devrim GÜNDÜZ
Hi, On Thu, 2007-06-07 at 13:12 +0200, Peter Eisentraut wrote: make dist[check] AFAIK make dist does not prepare postgres.tar.gz. Regards, -- Devrim GÜNDÜZ PostgreSQL Replication, Consulting, Custom Development, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP,

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Heikki Linnakangas
Thinking about this whole idea a bit more, it occured to me that the current approach to write all, then fsync all is really a historical artifact of the fact that we used to use the system-wide sync call instead of fsyncs to flush the pages to disk. That might not be the best way to do things

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Thinking about this whole idea a bit more, it occured to me that the current approach to write all, then fsync all is really a historical artifact of the fact that we used to use the system-wide sync call instead of fsyncs to flush the pages to

Re: [HACKERS] How do we create the releases?

2007-06-07 Thread Marc G. Fournier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Thursday, June 07, 2007 14:22:22 +0300 Devrim GÜNDÜZ [EMAIL PROTECTED] wrote: Hi, On Thu, 2007-06-07 at 13:12 +0200, Peter Eisentraut wrote: make dist[check] AFAIK make dist does not prepare postgres.tar.gz. right, as peter

[HACKERS] little PITR annoyance

2007-06-07 Thread ohp
Hi all, I've benn biten twice by this one: I enabled archiving on both my pgsql servers. the archive command was a simple gzip on an nfs mounted dir. Now the nfs server crashed. Both wal space filled up until postgresql shuts itself down because of no more space for WAL. That perfectly normal and

Re: [HACKERS] Vacuuming anything zeroes shared table stats

2007-06-07 Thread Alvaro Herrera
Michael Fuhr wrote: Is vacuuming any table supposed to zero the statistics for all shared tables? Doesn't that have implications for autovacuum? The example below is in 8.2.4 but I'm seeing similar behavior in 8.1.9 and 8.3devel. The problem is that the database hash is cleared of databases

Re: [HACKERS] Vacuuming anything zeroes shared table stats

2007-06-07 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: The problem is that the database hash is cleared of databases that no longer exist, and the database list is constructed by scanning pg_database. Since no entry exist for the database we use for shared tables (InvalidOid), the hash table is dropped.

Re: [HACKERS] [COMMITTERS] pgsql: Create a GUC parametertemp_tablespacesthat allows selection of

2007-06-07 Thread Tom Lane
Jaime Casanova [EMAIL PROTECTED] writes: On 6/4/07, Tom Lane [EMAIL PROTECTED] wrote: Perhaps a reasonable compromise could work like this: at the first point in a transaction where a temp file is created, choose a random list element, and thereafter advance cyclically for the duration of that

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-07 Thread Jim C. Nasby
On Mon, Jun 04, 2007 at 11:04:26AM -0400, Alvaro Herrera wrote: The launcher is set up to wake up in autovacuum_naptime seconds at most. So if the user configures a ridiculuos time (for example 86400 seconds, which I've seen) then the launcher would not detect the postmaster death Yeah, I've

Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-07 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: But this is misleading (started postmaster with good value, then edited postgresql.conf and entered -2): 17903 LOG: received SIGHUP, reloading configuration files 17903 LOG: -2 is outside the valid range for parameter

Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-07 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: But this is misleading (started postmaster with good value, then edited postgresql.conf and entered -2): 17903 LOG: received SIGHUP, reloading configuration files 17903 LOG: -2 is outside the valid range for

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Thinking about this whole idea a bit more, it occured to me that the current approach to write all, then fsync all is really a historical artifact of the fact that we used to use the system-wide sync call instead of fsyncs to flush

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: I don't think it's a historical artifact at all: it's a valid reflection of the fact that we don't know enough about disk layout to do low-level I/O scheduling. Issuing more fsyncs than necessary will do little except guarantee a

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: I don't think it's a historical artifact at all: it's a valid reflection of the fact that we don't know enough about disk layout to do low-level I/O scheduling. Issuing more fsyncs than necessary will do little

Re: [HACKERS] Vacuuming anything zeroes shared table stats

2007-06-07 Thread Michael Fuhr
On Thu, Jun 07, 2007 at 11:41:56AM -0400, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: The attached patch fixes this. Wouldn't it be easier to just special-case the shared DB in pgstat_vacuum_tabstat? Thanks; I'll test these patches when I get a chance. Additionally, in

Re: [HACKERS] Vacuuming anything zeroes shared table stats

2007-06-07 Thread Tom Lane
Michael Fuhr [EMAIL PROTECTED] writes: Further investigation shows that what really seems to be happening in 8.3devel is that the statistics for shared tables are reset every 15 seconds when autovacuum is enabled, which it is by default. I don't observe this phenomenon when autovacuum is

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Greg Smith
On Thu, 7 Jun 2007, Heikki Linnakangas wrote: So there's two extreme ways you can use LDC: 1. Finish the checkpoint as soon as possible, without disturbing other activity too much 2. Disturb other activity as little as possible, as long as the checkpoint finishes in a reasonable time. Are

Re: [HACKERS] Vacuuming anything zeroes shared table stats

2007-06-07 Thread Alvaro Herrera
Tom Lane wrote: Michael Fuhr [EMAIL PROTECTED] writes: Further investigation shows that what really seems to be happening in 8.3devel is that the statistics for shared tables are reset every 15 seconds when autovacuum is enabled, which it is by default. I don't observe this phenomenon

Re: [HACKERS] [PATCHES] [BUGS] BUG #3326: Invalid lower bound of autovacuum_cost_limit

2007-06-07 Thread Alvaro Herrera
Matthew T. O'Connor wrote: Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: But this is misleading (started postmaster with good value, then edited postgresql.conf and entered -2): 17903 LOG: received SIGHUP, reloading configuration files 17903 LOG: -2 is

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-07 Thread Andrew Hammond
On 6/7/07, Jim C. Nasby [EMAIL PROTECTED] wrote: On Mon, Jun 04, 2007 at 11:04:26AM -0400, Alvaro Herrera wrote: The launcher is set up to wake up in autovacuum_naptime seconds at most. So if the user configures a ridiculuos time (for example 86400 seconds, which I've seen) then the launcher

[HACKERS] pqlib suggestion

2007-06-07 Thread Dann Corbit
Clients using pqlib can get some boost by increasing tcp/ip window size. It might be good to make it a settable parameter. Anyway, maybe something like this: /* -- * connectNoDelay - * Sets the TCP_NODELAY socket option. * Returns 1 if successful, 0 if not. * -- */

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-07 Thread Tom Lane
Andrew Hammond [EMAIL PROTECTED] writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. It seems like we should have an upper limit on the GUC

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Gregory Stark
Greg Smith [EMAIL PROTECTED] writes: I'm completely biased because of the workloads I've been dealing with recently, but I consider (2) so much easier to tune for that it's barely worth worrying about. If your system is so underloaded that you can let the checkpoints take their own sweet

Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-07 Thread Matthew T. O'Connor
Tom Lane wrote: Andrew Hammond [EMAIL PROTECTED] writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the naptime really high is a Bad Idea. It seems like we should have an upper

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Greg Smith
On Thu, 7 Jun 2007, Gregory Stark wrote: You seem to have imagined that letting the checkpoint take longer will slow down transactions. And you seem to have imagined that I have so much spare time that I'm just making stuff up to entertain myself and sow confusion. I observed some

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-07 Thread Florian G. Pflug
Simon Riggs wrote: On Wed, 2007-06-06 at 16:11 +0200, Florian G. Pflug wrote: .) Added a new GUC operational_mode, which can be set to either readwrite or readonly. If it is set to readwrite (the default), postgres behaves as usual. All the following changes are only in effect if

Re: [HACKERS] Controlling Load Distributed Checkpoints

2007-06-07 Thread Joshua D. Drake
This is really a serious issue with the current design of the database, one that merely changes instead of going away completely if you throw more hardware at it. I'm perversely glad to hear this is torturing more people than just me as it improves the odds the situation will improve. It

Re: [HACKERS] Minor changes to Recovery related code

2007-06-07 Thread Simon Riggs
On Sat, 2007-03-31 at 00:51 +0200, Florian G. Pflug wrote: Simon Riggs wrote: On Fri, 2007-03-30 at 16:34 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: 2. pg_stop_backup() should wait until all archive files are safely archived before returning Not sure I agree with that

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-07 Thread Florian G. Pflug
Heikki Linnakangas wrote: Florian G. Pflug wrote: Jeff Davis wrote: Are you referring to the size of the xip array being a problem? Would it help to tie the size of the xip array to max_connections? I understand that max_connections might be greater on the master, but maybe something similar?

Re: [HACKERS] [COMMITTERS] pgsql: Avoid losing track of data for shared tables in pgstats.

2007-06-07 Thread Gregory Stark
Alvaro Herrera [EMAIL PROTECTED] writes: Log Message: --- Avoid losing track of data for shared tables in pgstats. Report by Michael Fuhr, patch from Tom Lane after a messier suggestion by me. When does this bug date to? is it possible it's related to the performance drop

Re: [HACKERS] [RFC] GSoC Work on readonly queries done so far

2007-06-07 Thread Florian G. Pflug
Jeff Davis wrote: On Wed, 2007-06-06 at 22:36 +0100, Simon Riggs wrote: .) Transactions are assigned a dummy xid ReadOnlyTransactionId, that is considered to be later than any other xid. So you are bumping FirstNormalTransactionId up by one for this? You're assuming then that we will

Re: [HACKERS] [ADMIN] Attempt to re-archive existing WAL logsafterrestoringfrom backup

2007-06-07 Thread Simon Riggs
On Thu, 2007-06-07 at 22:20 +0100, Simon Riggs wrote: On Sat, 2007-06-02 at 01:07 +0100, Jon Colverson wrote: Simon Riggs wrote: I've spent some time re-examining the code and I can't see a route for the discussed problem to occur. All the main code paths do not exhibit this problem in

Re: [HACKERS] [COMMITTERS] pgsql: Avoid losing track of data for shared tables in pgstats.

2007-06-07 Thread Alvaro Herrera
Gregory Stark wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Log Message: --- Avoid losing track of data for shared tables in pgstats. Report by Michael Fuhr, patch from Tom Lane after a messier suggestion by me. When does this bug date to? It was in 8.1. I didn't

Re: [HACKERS] little PITR annoyance

2007-06-07 Thread Simon Riggs
On Thu, 2007-06-07 at 17:01 +0200, [EMAIL PROTECTED] wrote: I've benn biten twice by this one: I enabled archiving on both my pgsql servers. the archive command was a simple gzip on an nfs mounted dir. Now the nfs server crashed. Both wal space filled up until postgresql shuts itself down

Best Practice for running vacuums during off hours WAS Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-07 Thread Andrew Hammond
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/7/07, Matthew T. O'Connor wrote: Tom Lane wrote: Andrew Hammond writes: Hmmm... it seems to me that points new users towards not using autovacuum, which doesn't seem like the best idea. I think it'd be better to say that setting the

Re: Best Practice for running vacuums during off hours WAS Re: [HACKERS] Autovacuum launcher doesn't notice death of postmaster immediately

2007-06-07 Thread Alvaro Herrera
Andrew Hammond escribió: That's a good question. I can't see any reason for a naptime longer than 60 seconds either. I think very large naptime settings are a symptom of another issue: what's the Right Way to defer vacuums until off hours? Is that even a desirable thing anymore? I don't

Re: [HACKERS] [COMMITTERS] pgsql: Avoid losing track of data for shared tables in pgstats.

2007-06-07 Thread Gregory Stark
Alvaro Herrera [EMAIL PROTECTED] writes: Gregory Stark wrote: is it possible it's related to the performance drop immediately following a vacuum analyze we've been seeing? I don't think so, unless you were counting on pgstats data of shared tables for something. The optimizer, for one,

Re: [HACKERS] TOAST usage setting

2007-06-07 Thread Bruce Momjian
Gregory Stark wrote: Bruce Momjian [EMAIL PROTECTED] writes: Well, it is summarized here: http://momjian.us/expire/TOAST/SUMMARY.html It made non-TOAST access 2x faster, but TOAST 7x slower, and that seemed like a good compromise. Is this still testing with all data fitting

Re: [HACKERS] .conf File Organization WAS: Controlling Load Distributed Checkpoints

2007-06-07 Thread Josh Berkus
All, This brings up another point. With the increased number of .conf options, the file is getting hard to read again. I'd like to do another reorganization, but I don't really want to break people's diff scripts. Should I worry about that? --Josh ---(end of

Re: [HACKERS] .conf File Organization WAS: Controlling Load Distributed Checkpoints

2007-06-07 Thread Joshua D. Drake
Josh Berkus wrote: All, This brings up another point. With the increased number of .conf options, the file is getting hard to read again. I'd like to do another reorganization, but I don't really want to break people's diff scripts. Should I worry about that? As a point of feedback,

Re: [HACKERS] [COMMITTERS] pgsql: Avoid losing track of data for shared tables in pgstats.

2007-06-07 Thread Alvaro Herrera
Gregory Stark wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Gregory Stark wrote: is it possible it's related to the performance drop immediately following a vacuum analyze we've been seeing? I don't think so, unless you were counting on pgstats data of shared tables for

Re: [HACKERS] .conf File Organization WAS: Controlling Load Distributed Checkpoints

2007-06-07 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: This brings up another point. With the increased number of .conf options, the file is getting hard to read again. I'd like to do another reorganization, but I don't really want to break people's diff scripts. Do you have a better organizing principle

Re: [HACKERS] [COMMITTERS] pgsql: Avoid losing track of data for shared tables in pgstats.

2007-06-07 Thread Michael Fuhr
On Thu, Jun 07, 2007 at 11:46:17PM +0100, Gregory Stark wrote: Does anything use the pgstats data for anything other than presenting feedback to users? Autovacuum uses it to estimate when tables should be vacuumed right? This wouldn't have caused autovacuum to go nuts vacuuming these tables

Re: [HACKERS] [COMMITTERS] pgsql: Avoid losing track of data for shared tables in pgstats.

2007-06-07 Thread Michael Fuhr
On Thu, Jun 07, 2007 at 06:09:36PM -0400, Alvaro Herrera wrote: Gregory Stark wrote: When does this bug date to? It was in 8.1. I didn't verify whether it affects on 8.0; I think we separated the shared tables in pgstats in 8.1, with the advent of autovacuum, so I assume it doesn't.

Re: [HACKERS] [COMMITTERS] pgsql: Avoid losing track of data for shared tables in pgstats.

2007-06-07 Thread Tom Lane
Michael Fuhr [EMAIL PROTECTED] writes: On Thu, Jun 07, 2007 at 06:09:36PM -0400, Alvaro Herrera wrote: Gregory Stark wrote: When does this bug date to? It was in 8.1. I didn't verify whether it affects on 8.0; I think we separated the shared tables in pgstats in 8.1, with the advent of

Re: [HACKERS] TOAST usage setting

2007-06-07 Thread Gregory Stark
Bruce Momjian [EMAIL PROTECTED] writes: Gregory Stark wrote: Is this still testing with all data fitting in RAM? Yes. Having things out of RAM is going to make access even slower, but it is going to allow the heap to be in RAM more often. It would let us measure the actual impact of