Re: [GENERAL] pg_upgrade from 9.0.7 to 9.1.3: duplicate key pg_authid_oid_index

2012-06-01 Thread Bryan Murphy
On Thu, May 31, 2012 at 4:28 PM, Jeff Davis pg...@j-davis.com wrote: On Thu, 2012-05-31 at 15:55 -0500, Bryan Murphy wrote: I'm having a problem upgrading a cluster from 9.0.7 to 9.1.3. Here's the error: Please send /srv/pg_upgrade_dump_globals.sql Also, can you restart the old system

Re: [GENERAL] pg_upgrade from 9.0.7 to 9.1.3: duplicate key pg_authid_oid_index

2012-06-01 Thread Bryan Murphy
On Fri, Jun 1, 2012 at 8:07 AM, Bryan Murphy bmurphy1...@gmail.com wrote: On Thu, May 31, 2012 at 4:28 PM, Jeff Davis pg...@j-davis.com wrote: On Thu, 2012-05-31 at 15:55 -0500, Bryan Murphy wrote: I'm having a problem upgrading a cluster from 9.0.7 to 9.1.3. Here's the error: Please

[GENERAL] pg_upgrade from 9.0.7 to 9.1.3: duplicate key pg_authid_oid_index

2012-05-31 Thread Bryan Murphy
I'm having a problem upgrading a cluster from 9.0.7 to 9.1.3. Here's the error: psql:/srv/pg_upgrade_dump_globals.sql:54: ERROR: duplicate key value violates unique constraint pg_authid_oid_index DETAIL: Key (oid)=(10) already exists. Any ideas what I'm doing wrong? Here's the verbose output

[GENERAL] Hit by the out of memory killer last night

2011-01-31 Thread Bryan Murphy
Last night we were hit by the out of memory killer. Looking at the following graph, you can clearly see unusual memory growth. This is a database server running Postgres 9.0.0. http://mediafly-public.s3.amazonaws.com/dbcluster02-master-month.png We have another server, running Postgres 9.0.1

Re: [GENERAL] Hit by the out of memory killer last night

2011-01-31 Thread Bryan Murphy
On Mon, Jan 31, 2011 at 10:35 AM, Ben Chobot be...@silentmedia.com wrote: Any advice? What should I be looking for? Any particular reason you are running the OOM killer on a database server? Why have the kernel set to overcommit memory in the first place? Simply an oversight. That being

Re: [GENERAL] pgpool-II 3.0 + postgres 9rc1 + md5 authentication not working

2010-09-21 Thread Bryan Murphy
On Mon, Sep 20, 2010 at 6:23 PM, Tatsuo Ishii is...@sraoss.co.jp wrote: I have used PostgreSQL 9.0 + pgpool-II 3.0 and they work fine with md5 auth. Your log seems to indicate that the password in pool_passwd and the one in pg_shadow are not identical. Can you verify that? The query result:

Re: [GENERAL] pgpool-II 3.0 + postgres 9rc1 + md5 authentication not working

2010-09-21 Thread Bryan Murphy
On Tue, Sep 21, 2010 at 10:26 AM, Bryan Murphy bmurphy1...@gmail.comwrote: On Mon, Sep 20, 2010 at 6:23 PM, Tatsuo Ishii is...@sraoss.co.jp wrote: I have used PostgreSQL 9.0 + pgpool-II 3.0 and they work fine with md5 auth. Your log seems to indicate that the password in pool_passwd

Re: [GENERAL] pgpool-II 3.0 + postgres 9rc1 + md5 authentication not working

2010-09-21 Thread Bryan Murphy
On Tue, Sep 21, 2010 at 10:45 AM, Bryan Murphy bmurphy1...@gmail.comwrote: I'm sorry, when I went back over to double check my steps I realized I ran the wrong command. I am *still* having the problem. It appears that the MD5 hashes now match, but it's still failing. I have postgres

Re: [GENERAL] pgpool-II 3.0 + postgres 9rc1 + md5 authentication not working

2010-09-21 Thread Bryan Murphy
On Tue, Sep 21, 2010 at 8:08 PM, Tatsuo Ishii is...@postgresql.org wrote: Unfortunately the gdb backtrace does not show enough information because of optimization, I guess. Can you take a backtrace with optimization disabled binary? You can obtain this by editing Makefile around line 147.

Re: [GENERAL] pgpool-II 3.0 + postgres 9rc1 + md5 authentication not working

2010-09-20 Thread Bryan Murphy
On Sun, Sep 19, 2010 at 11:31 PM, Tatsuo Ishii is...@sraoss.co.jp wrote: Sorry for delay. I had a trip outside Japan. No problem. I found nasty bug with pgpool. Please try attached patches. I tried the patch file and I still cannot connect. The only other difference is that I've already

Re: [GENERAL] pgpool-II 3.0 + postgres 9rc1 + md5 authentication not working

2010-09-15 Thread Bryan Murphy
On Tue, Sep 14, 2010 at 6:55 PM, Tatsuo Ishii is...@sraoss.co.jp wrote: Sorry for not enough description about pool_passwd. It's located under the same directory as pgpool.conf. So the default is /usr/local/etc/pool_passwd. You need to create /usr/local/etc/pool_passwd if the uid to run

[GENERAL] pgpool-II 3.0 + postgres 9rc1 + md5 authentication not working

2010-09-14 Thread Bryan Murphy
I can't get md5 authentication working with postgres 9rc1 and pgpool-II 3.0. I see references to pool_passwd in the pgpool documentation, but I see nothing indicating *where* this file should exist and how pgpool finds it. I've set my accounts up in pcp.conf, however, I do not believe this is

Re: [GENERAL] missing chunk number 0 for toast value 25693266 in pg_toast_25497233

2010-05-07 Thread Bryan Murphy
On Fri, May 7, 2010 at 9:02 AM, Magnus Hagander mag...@hagander.net wrote: Try doing a binary search with LIMIT. E.g., if you have 20M reecords, do a SELECT * FROM ... LIMIT 10M. (throw away the results) If that broke, check the upper half, if not, check the lower one (with OFFSET). If you

[GENERAL] missing chunk number 0 for toast value 25693266 in pg_toast_25497233

2010-05-06 Thread Bryan Murphy
I'm running into this issue again: psql --version psql (PostgreSQL) 8.3.7 COPY items_extended TO '/dev/null'; ERROR: missing chunk number 0 for toast value 25693266 in pg_toast_25497233 Unfortunately, I do not know where these are coming from and I cannot replicate the data in at least one of

[GENERAL] Postgres 9.0 Hot Standby + Fail Over

2010-05-04 Thread Bryan Murphy
We have a production database that contains data which is easily recreated at runtime. I'm considering upgrading this to 9.0 beta1 to get some experience with the new hot standby system on a server that is under medium to heavy load. Obviously, being a production database, it's inconvenient if

Re: [GENERAL] Postgresql on EC2/EBS in production?

2010-04-28 Thread Bryan Murphy
On Tue, Apr 27, 2010 at 11:32 PM, Greg Smith g...@2ndquadrant.com wrote: What do you mean by an instance failure here?  The actual EC2 image getting corrupted so that it won't boot anymore, or just the instance going down badly? The instance going down, badly. The last time it happened, what

Re: [GENERAL] Postgresql on EC2/EBS in production?

2010-04-27 Thread Bryan Murphy
On Tue, Apr 27, 2010 at 11:31 AM, Greg Smith g...@2ndquadrant.com wrote: Nikhil G. Daddikar wrote: I was wondering if any of you are using (or tried to use) PG+EC2/EBS on a production system. Are any best-practices. Googling didn't help much. A few articles I came across scared me a bit.

Re: [GENERAL] Warm Standby Setup Documentation

2010-03-26 Thread Bryan Murphy
On Fri, Mar 26, 2010 at 1:32 PM, Greg Smith g...@2ndquadrant.com wrote: If there's another server around, you can have your archive_command on the master ship to two systems, then use the second one as a way to jump-start this whole process. After fail-over, just start shipping from the new

Re: [GENERAL] Warm Standby Setup Documentation

2010-03-24 Thread Bryan Murphy
On Mon, Mar 22, 2010 at 9:21 AM, Ogden li...@darkstatic.com wrote: I have looked all over but could not find any detailed docs on setting up a warm standby solution using PostgreSQL 8.4. I do know of http://www.postgresql.org/docs/8.4/static/warm-standby.html but was wondering if there was a

[GENERAL] ERROR: attempted to delete invisible tuple

2009-08-17 Thread Bryan Murphy
We had a hardware failure last week and had to switch over to our spare. Unfortunately, at some point we managed to get some data corruption. I've been going through the database table by table, record by record, trying to find the problems and fix them. This one has me stumped. We have one

Re: [GENERAL] ERROR: attempted to delete invisible tuple

2009-08-17 Thread Bryan Murphy
On Mon, Aug 17, 2009 at 11:35 AM, Greg Stark gsst...@mit.edu wrote: On Mon, Aug 17, 2009 at 4:23 PM, Bryan Murphybmurphy1...@gmail.com wrote: I've identified 82 bad records. When I try to query for the records, we get the following: ERROR: missing chunk number 0 for toast value

Re: [GENERAL] ERROR: attempted to delete invisible tuple

2009-08-17 Thread Bryan Murphy
Could I run pg_resetxlog on a warm spare? Would that give the same result? Unfortunately, this is our production system and I simply cannot bring it down at the moment to run pg_resetxlog. Bryan On Mon, Aug 17, 2009 at 11:35 AM, Greg Stark gsst...@mit.edu wrote: On Mon, Aug 17, 2009 at 4:23

Re: [GENERAL] ERROR: attempted to delete invisible tuple

2009-08-17 Thread Bryan Murphy
On Mon, Aug 17, 2009 at 12:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bryan Murphy bmurphy1...@gmail.com writes: Here's the xmin/xmax/ctid for three problematic records: prodpublic=# select xmin,xmax,ctid from items_extended where id in ('34537ed90d7546d78f2c172fc8eed687

Re: [GENERAL] ERROR: attempted to delete invisible tuple

2009-08-17 Thread Bryan Murphy
On Mon, Aug 17, 2009 at 12:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bryan Murphy bmurphy1...@gmail.com writes: On Mon, Aug 17, 2009 at 12:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hm, what's your current XID counter? (pg_controldata would give an approximate answer.) I'm wondering

Re: [GENERAL] ERROR: attempted to delete invisible tuple

2009-08-17 Thread Bryan Murphy
On Mon, Aug 17, 2009 at 4:02 PM, Greg Stark gsst...@mit.edu wrote: For what it's worth at EDB I dealt with another case like this and I imagine others have too. I think it's too easy to do things in the wrong order or miss a step and end up with these kinds of problems. I would really like

Re: [GENERAL] Failover, Wal Logging, and Multiple Spares

2009-08-17 Thread Bryan Murphy
PM, Bryan Murphy bmurphy1...@gmail.com wrote: Assuming we are running a Postgres instance that is shipping log files to 2 or more warm spares, is there a way I can fail over to one of the spares, and have the second spare start receiving updates from the new master without missing a beat? I can

[GENERAL] Failover, Wal Logging, and Multiple Spares

2009-08-16 Thread Bryan Murphy
Assuming we are running a Postgres instance that is shipping log files to 2 or more warm spares, is there a way I can fail over to one of the spares, and have the second spare start receiving updates from the new master without missing a beat? I can live with losing the old master, and at least

[GENERAL] Having trouble restoring our backups

2009-06-12 Thread Bryan Murphy
Hey guys, I'm having difficulty restoring some of our backups. Luckily, I'm only trying to do this to bring up a copy of our database for testing purposes, but this still has me freaked out because it means we currently have no valid backups and are only running with a single warm spare. Our

Re: [GENERAL] Having trouble restoring our backups

2009-06-12 Thread Bryan Murphy
On Fri, Jun 12, 2009 at 10:48 AM, Alan Hodgson ahodg...@simkin.ca wrote: On Friday 12 June 2009, Bryan Murphy bmurphy1...@gmail.com wrote: What am I doing wrong? FYI, we're running 8.3.7. See the documentation on PITR backups for how to do this correctly. I've read through the PITR

Re: [GENERAL] Having trouble restoring our backups

2009-06-12 Thread Bryan Murphy
On Fri, Jun 12, 2009 at 11:08 AM, Bryan Murphy bmurphy1...@gmail.comwrote: I've read through the PITR documentation many times. I do not see anything that sheds light on what I'm doing wrong, and I've restored older backups successfully many times in the past few months using this technique

Re: [GENERAL] Minimizing Recovery Time (wal replication)

2009-04-13 Thread Bryan Murphy
On Sun, Apr 12, 2009 at 5:52 AM, Simon Riggs si...@2ndquadrant.com wrote: The database is performing too frequent restartpoints. This has been optimised in PostgreSQL 8.4 by the addition of the bgwriter running during recovery. This will mean that your hot spare will not pause while waiting

[GENERAL] Minimizing Recovery Time (wal replication)

2009-04-09 Thread Bryan Murphy
I have two hot-spare databases that use wal archiving and continuous recovery mode. I want to minimize recovery time when we have to fail over to one of our hot spares. Right now, I'm seeing the following behavior which makes a quick recovery seem problematic: (1) hot spare applies 70 to 75 wal

Re: [GENERAL] Minimizing Recovery Time (wal replication)

2009-04-09 Thread Bryan Murphy
On Thu, Apr 9, 2009 at 6:38 PM, Greg Smith gsm...@gregsmith.com wrote: What does vmstat say about the bi/bo during this time period?  It sounds like the volume of random I/O produced by recovery is just backing up as expected.  Some quick math: I'll have to capture this, unfortunately I won't

Re: [GENERAL] Minimizing Recovery Time (wal replication)

2009-04-09 Thread Bryan Murphy
On Thu, Apr 9, 2009 at 7:33 PM, Greg Smith gsm...@gregsmith.com wrote: 1) Decrease the maximum possible segment backlog so you can never get this   far behind I understand conceptually what you are saying, but I don't know how to practically realize this. :)  Do you mean lower

[GENERAL] Backup Strategy Second Opinion

2009-02-22 Thread Bryan Murphy
Hey guys, we just moved our system to Amazon's EC2 service. I'm a bit paranoid about backups, and this environment is very different than our previous environment. I was hoping you guys could point out any major flaws in our backup strategy that I may have missed. A few assumptions: 1. It's OK

Re: [GENERAL] Backup Strategy Second Opinion

2009-02-22 Thread Bryan Murphy
On Sun, Feb 22, 2009 at 7:30 PM, Tim Uckun timuc...@gmail.com wrote: 1. It's OK if we lose a few seconds (or even minutes) of transactions should one of our primary databases crash. 2. It's unlikely we'll need to load a backup that's more than a few days old. How do you handle failover and

[GENERAL] Schema Export/Import

2008-02-29 Thread Bryan Murphy
Hey guys, we're changing the way we version our database from some old unsupported (and crappy) ruby migrations-like methodology to straight SQL scripts. We currently run CruiseControl.NET on a windows machine pointed to a test database server hosted on linux for our builds. At each build we

[GENERAL] full text index and most frequently used words

2008-02-08 Thread Bryan Murphy
I'm a bit of a novice writing tsearch2 queries, so forgive me if this is a basic question. We have a table with 2million+ records which has a considerable amount of text content. Some search terms (such as comedy, new, news, music, etc.) cause a significant performance hit on our web site.

Re: [GENERAL] pg_dump and server responsiveness

2007-12-06 Thread Bryan Murphy
On Dec 5, 2007 9:49 PM, Tom Lane [EMAIL PROTECTED] wrote: Only access-share locks, but that could still be an issue if anything in your system likes to take exclusive locks. Have you looked into pg_locks to see if anything's getting blocked? pg_dump is entirely capable of causing an

Re: [GENERAL] pg_dump and server responsiveness

2007-12-06 Thread Bryan Murphy
On Dec 6, 2007 10:09 AM, Tom Lane [EMAIL PROTECTED] wrote: Why dump such a table at all? It evidently doesn't contain any data you need to preserve ... I forget which version you are running, but 8.2 pg_dump has an --exclude-table switch which'd work peachy for this. I did not know about

[GENERAL] pg_dump and server responsiveness

2007-12-05 Thread Bryan Murphy
When we run pg_dump on our database, our web site becomes completely unresponsive. I thought pg_dump was runnable while the database was still being actively used? Anyway, I'm not entirely sure why, but here's what I'm seeing. pg_dump -v database_name | gzip output_file 25% to 50% CPU usage (4

Re: [GENERAL] pg_dump and server responsiveness

2007-12-05 Thread Bryan Murphy
Sorry about the formatting, here's the dump as a text file. Thanks, Bryan On Dec 5, 2007 10:05 AM, Bryan Murphy [EMAIL PROTECTED] wrote: When we run pg_dump on our database, our web site becomes completely unresponsive. I thought pg_dump was runnable while the database was still being

Re: [GENERAL] pg_dump and server responsiveness

2007-12-05 Thread Bryan Murphy
On Dec 5, 2007 10:14 AM, Joshua D. Drake [EMAIL PROTECTED] wrote: Pg_dump uses Access Share if I recall. You can operate normally while running pg_dump. I am having a hard time parsing that. Could you instead go over to pgsql.privatepaste.com and send back a paste link?

Re: [GENERAL] Index Usage

2007-10-17 Thread Bryan Murphy
On 10/17/07, Joao Miguel Ferreira [EMAIL PROTECTED] wrote: If your intention is to eliminate the unused indexes rows you should run 'vaccum' and/or 'vacuum full' and/or 'reindex'. This also has the consequence of freing filesystem space and returning it back to the OS. Check it out here:

[GENERAL] Index Usage

2007-10-16 Thread Bryan Murphy
Is there a way I can track index usage over a long period of time? Specifically, I'd like to identify indexes that aren't being regularly used and drop them. Bryan ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [GENERAL] Optimising SELECT on a table with one million rows

2007-07-30 Thread Bryan Murphy
First question... did you create the appropriate indexes on the appropriate columns for these tables? Foreign keys do not implicitly create indexes in postgres. Bryan On 7/30/07, Cultural Sublimation [EMAIL PROTECTED] wrote: Hi, I'm fairly new with Postgresql, so I am not sure if the

Re: [GENERAL] about c# and postgresql

2007-07-23 Thread Bryan Murphy
I highly recommend you use the Npgsql driver, and if you're feeling really saucy try NHibernate on top of that. http://pgfoundry.org/projects/npgsql http://www.nhibernate.org/ Bryan On 7/23/07, longlong [EMAIL PROTECTED] wrote: hi,all i have a local system with windows xp. i want to use c#

Re: [GENERAL] Deleted Flag/Unique Constraint

2007-04-02 Thread Bryan Murphy
where I have yet to find a lot of guidance on the issue. Bryan On 3/29/07, Lew [EMAIL PROTECTED] wrote: Bryan Murphy wrote: I think the other guys suggestion will work better. ;) Really, the table was just an example off the top of my head. I believe we do use a boolean as the deleted flag. We

[GENERAL] Deleted Flag/Unique Constraint

2007-03-29 Thread Bryan Murphy
Is it possible to declare a unique constraint in combination with a deleted flag? For example, if I have a table like this: CREATE TABLE ( ID NOT NULL PRIMARY KEY, Key VARCHAR(32) NOT NULL, Value VARCHAR(32) NOT NULL, Deleted INT NOT NULL DEFAULT 0 ); can I declare a unique constraint that

Re: [GENERAL] Deleted Flag/Unique Constraint

2007-03-29 Thread Bryan Murphy
Thanks! That works great! Bryan On 3/29/07, Jonathan Hedstrom [EMAIL PROTECTED] wrote: Bryan Murphy wrote: Is it possible to declare a unique constraint in combination with a deleted flag? For example, if I have a table like this: CREATE TABLE ( ID NOT NULL PRIMARY KEY, Key

Re: [GENERAL] Deleted Flag/Unique Constraint

2007-03-29 Thread Bryan Murphy
, Bryan Murphy wrote: Is it possible to declare a unique constraint in combination with a deleted flag? For example, if I have a table like this: CREATE TABLE ( ID NOT NULL PRIMARY KEY, Key VARCHAR(32) NOT NULL, Value VARCHAR(32) NOT NULL, Deleted INT NOT NULL DEFAULT 0 ); can